2016-05-17 21:29:18,395 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(197)) - found 1 test platforms from classpath
2016-05-17 21:29:18,419 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(246)) - adding test: cascading.detail.EveryPipeAssemblyPlatformTest, with platform: hadoop2-tez
2016-05-17 21:29:18,793 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-05-17 21:29:19,085 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-05-17 21:29:19,433 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(75)) - platform property overrides:
2016-05-17 21:29:19,623 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:19,624 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:19,755 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: 147ECB3F3C44467E9943797220AECA7D
2016-05-17 21:29:19,960 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.083
2016-05-17 21:29:19,980 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.104
2016-05-17 21:29:19,982 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:19,983 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:19,984 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:20,161 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:20,299 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_-1
2016-05-17 21:29:20,494 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.1.0
2016-05-17 21:29:20,497 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:20,497 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:20,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_-1"]
2016-05-17 21:29:20,498 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:20,499 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:20,499 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:20,500 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...blyplatform/-1_'field'_-1
2016-05-17 21:29:20,500 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:20,521 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:20,696 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:20,702 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:20,702 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:20,748 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C01539B622544ACC99C0AE01B7C83D2D/ACB2310AB3344A7890BC22B6A7F7F508/.tez/application_1463520560702_0001 doesn't exist and is created
2016-05-17 21:29:20,820 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:20,820 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:20,821 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C01539B622544ACC99C0AE01B7C83D2D/ACB2310AB3344A7890BC22B6A7F7F508/.tez/application_1463520560702_0001
2016-05-17 21:29:20,864 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520560702_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:20,914 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:20,920 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-05-17 21:29:20,920 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-05-17 21:29:20,921 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:20,929 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:20,953 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:20,957 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:20,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C01539B622544ACC99C0AE01B7C83D2D/ACB2310AB3344A7890BC22B6A7F7F508/.tez/application_1463520560702_0001/localmode-log-dir
2016-05-17 21:29:20,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C01539B622544ACC99C0AE01B7C83D2D/ACB2310AB3344A7890BC22B6A7F7F508/.tez/application_1463520560702_0001/localmode-log-dir/history.txt.appattempt_1463520560702_0001_000000, maxErrors=10
2016-05-17 21:29:20,961 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:20,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520560702_0001_000000, appSubmitTime=1463520560844, launchTime=1463520560863
2016-05-17 21:29:20,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:20,971 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:20,974 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:20,974 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:21,021 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 21:29:21,021 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 21:29:21,021 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:21,034 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49921
2016-05-17 21:29:21,078 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:21,122 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-05-17 21:29:21,122 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-05-17 21:29:21,122 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:21,160 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520560702_0001_000000, startTime=1463520561159
2016-05-17 21:29:21,160 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:21,160 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:21,222 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:21,239 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:21,239 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/-1_'field'_-1, applicationId=application_1463520560702_0001, dagName=[C01539B622544ACC99C0AE01B7C83D2D/ACB2310AB3344A7890BC22B6A7F7F508] every/(1/1) ...blyplatform/-1_'field'_-1
2016-05-17 21:29:21,495 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C01539B622544ACC99C0AE01B7C83D2D/ACB2310AB3344A7890BC22B6A7F7F508] every/(1/1) ...blyplatform/-1_'field'_-1
2016-05-17 21:29:21,627 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520560702_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C01539B622544ACC99C0AE01B7C83D2D/ACB2310AB3344A7890BC22B6A7F7F508/.tez/application_1463520560702_0001/localmode-log-dir/dag_1463520560702_0001_1.dot
2016-05-17 21:29:21,631 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C01539B622544ACC99C0AE01B7C83D2D/ACB2310AB3344A7890BC22B6A7F7F508/.tez/application_1463520560702_0001/localmode-log-dir/dag_1463520560702_0001_1-tez-dag.pb.txt
2016-05-17 21:29:21,757 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520560702_0001_1
2016-05-17 21:29:21,757 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:21,758 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C01539B622544ACC99C0AE01B7C83D2D/ACB2310AB3344A7890BC22B6A7F7F508] every/(1/1) ...blyplatform/-1_'field'_-1
2016-05-17 21:29:21 Running Dag: dag_1463520560702_0001_1
2016-05-17 21:29:21,800 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:21,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520560702_0001_1, submitTime=1463520561495
2016-05-17 21:29:21,838 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex ABD6007A8D6E4EBA94C0A316630DFA00
2016-05-17 21:29:21,847 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:21,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:21,849 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520560702_0001_1, initTime=1463520561806
2016-05-17 21:29:21,851 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520560702_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:21,855 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:21,855 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:DAG_STARTED]: dagID=dag_1463520560702_0001_1, startTime=1463520561854
2016-05-17 21:29:21,856 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:21,859 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520560702_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:21,860 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B7362DDF9FA7463399DCB5FA7D16FC24 : {627572587E81491DACEB3829A469FF60={InputName=627572587E81491DACEB3829A469FF60}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:21,860 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 627572587E81491DACEB3829A469FF60, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:21,861 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24]
2016-05-17 21:29:21,865 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/-1_'field'_-1, applicationId=application_1463520560702_0001, dagName=[C01539B622544ACC99C0AE01B7C83D2D/ACB2310AB3344A7890BC22B6A7F7F508] every/(1/1) ...blyplatform/-1_'field'_-1
2016-05-17 21:29:21,867 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24]
2016-05-17 21:29:21,867 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24]
2016-05-17 21:29:21,870 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24]. Starting root input initializers: 1
2016-05-17 21:29:21,877 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 627572587E81491DACEB3829A469FF60 on vertex vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24]
2016-05-17 21:29:21,891 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:21,895 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00]
2016-05-17 21:29:21,943 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:21,948 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 627572587E81491DACEB3829A469FF60 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:21,953 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:21,954 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:21,969 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520560702_0001, with dag id: dag_1463520560702_0001_1
2016-05-17 21:29:21,980 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:21,980 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00]
2016-05-17 21:29:22,000 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00]
2016-05-17 21:29:22,002 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00]
2016-05-17 21:29:22,003 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D74B710FE1F747DE84A9A6829102573C, vertexId=vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:22,007 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D74B710FE1F747DE84A9A6829102573C, vertexId=vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00]
2016-05-17 21:29:22,014 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:22,016 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 627572587E81491DACEB3829A469FF60 on vertex vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24]
2016-05-17 21:29:22,025 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D74B710FE1F747DE84A9A6829102573C, vertexName=ABD6007A8D6E4EBA94C0A316630DFA00, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:22,037 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D74B710FE1F747DE84A9A6829102573C, vertexId=vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00]
2016-05-17 21:29:22,040 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:22,041 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ABD6007A8D6E4EBA94C0A316630DFA00, vertexId=vertex_1463520560702_0001_1_01, initRequestedTime=1463520561892, initedTime=1463520562040, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:22,042 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:22,046 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:22,047 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {627572587E81491DACEB3829A469FF60=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:22,047 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24] parallelism set to 1
2016-05-17 21:29:22,047 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:22,055 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520560702_0001_1_00, eventCount=1
2016-05-17 21:29:22,057 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:22,058 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B7362DDF9FA7463399DCB5FA7D16FC24, vertexId=vertex_1463520560702_0001_1_00, initRequestedTime=1463520561860, initedTime=1463520562057, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:22,058 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24] with distanceFromRoot: 0
2016-05-17 21:29:22,058 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:22,059 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B7362DDF9FA7463399DCB5FA7D16FC24
2016-05-17 21:29:22,059 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24]
2016-05-17 21:29:22,061 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:22,061 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520560702_0001_1_00, startRequestedTime=1463520562058, startedTime=1463520562058
2016-05-17 21:29:22,062 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:22,092 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:22,093 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:TASK_STARTED]: vertexName=B7362DDF9FA7463399DCB5FA7D16FC24, taskId=task_1463520560702_0001_1_00_000000, scheduledTime=1463520562091, launchTime=1463520562091
2016-05-17 21:29:22,094 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520560702_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:22,094 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520560702_0001_1_00 for vertex: vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00] numStartedSources: 1 numSources: 1
2016-05-17 21:29:22,095 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00] with distanceFromRoot: 1
2016-05-17 21:29:22,100 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24]
2016-05-17 21:29:22,105 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B7362DDF9FA7463399DCB5FA7D16FC24 in vertex: ABD6007A8D6E4EBA94C0A316630DFA00
2016-05-17 21:29:22,108 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: ABD6007A8D6E4EBA94C0A316630DFA00 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:22,108 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:22,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520560702_0001_1_01, startRequestedTime=1463520562101, startedTime=1463520562101
2016-05-17 21:29:22,109 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:22,116 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C01539B622544ACC99C0AE01B7C83D2D/ACB2310AB3344A7890BC22B6A7F7F508] every/(1/1) ...blyplatform/-1_'field'_-1, VertexName: B7362DDF9FA7463399DCB5FA7D16FC24, VertexParallelism: 1, TaskAttemptID:attempt_1463520560702_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=627572587E81491DACEB3829A469FF60, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=ABD6007A8D6E4EBA94C0A316630DFA00, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:22,121 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520560702_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:22,121 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:22,161 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:22,164 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:22,164 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:22,169 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520560702_0001_00_000001
2016-05-17 21:29:22,169 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520560702_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:22,180 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520560702_0001_00_000001
2016-05-17 21:29:22,180 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:22,186 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:22,186 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520560702_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:22,187 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520560702_0001_00_000001, launchTime=1463520562186
2016-05-17 21:29:22,190 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520560702_0001_00_000001
2016-05-17 21:29:22,194 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520560702_0001_1_00_000000_0] to container: [container_1463520560702_0001_00_000001]
2016-05-17 21:29:22,194 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520560702_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:22,197 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520560702_0001_00_000001 given task: attempt_1463520560702_0001_1_00_000000_0
2016-05-17 21:29:22,197 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520560702_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520560702_0001_1_00_000000_0
2016-05-17 21:29:22,198 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520560702_0001_1_00_000000_0] started. Is using containerId: [container_1463520560702_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:22 Starting to run new task attempt: attempt_1463520560702_0001_1_00_000000_0
2016-05-17 21:29:22,198 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520560702_0001_1_00_000000_0
2016-05-17 21:29:22,198 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:22,200 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:22,202 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:22,202 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:22,203 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:22,204 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:22,205 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B7362DDF9FA7463399DCB5FA7D16FC24, taskAttemptId=attempt_1463520560702_0001_1_00_000000_0, startTime=1463520562196, containerId=container_1463520560702_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520560702_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:22,206 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520560702_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:22,207 INFO counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters: GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-05-17 21:29:22,210 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520560702_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:22,228 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:22,228 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C01539B622544ACC99C0AE01B7C83D2D/ACB2310AB3344A7890BC22B6A7F7F508] every/(1/1) ...blyplatform/-1_'field'_-1, VertexName: B7362DDF9FA7463399DCB5FA7D16FC24, VertexParallelism: 1, TaskAttemptID:attempt_1463520560702_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=627572587E81491DACEB3829A469FF60, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=ABD6007A8D6E4EBA94C0A316630DFA00, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:22,231 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:22,235 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520560702_0001_1_00_000000_0
2016-05-17 21:29:22,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:22,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:22,238 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520560702_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:22,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=627572587E81491DACEB3829A469FF60, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:22,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:22,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=ABD6007A8D6E4EBA94C0A316630DFA00, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:22,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:22,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 627572587E81491DACEB3829A469FF60
2016-05-17 21:29:22,246 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:22,249 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 627572587E81491DACEB3829A469FF60
2016-05-17 21:29:22,252 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: ABD6007A8D6E4EBA94C0A316630DFA00
2016-05-17 21:29:22,264 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:22,264 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: ABD6007A8D6E4EBA94C0A316630DFA00
2016-05-17 21:29:22,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: ABD6007A8D6E4EBA94C0A316630DFA00
2016-05-17 21:29:22,268 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:22,274 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:22,275 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 627572587E81491DACEB3829A469FF60
2016-05-17 21:29:22,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 627572587E81491DACEB3829A469FF60
2016-05-17 21:29:22,286 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B7362DDF9FA7463399DCB5FA7D16FC24, ordinal: 0
2016-05-17 21:29:22,287 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B7362DDF9FA7463399DCB5FA7D16FC24, mem on start (mb), free: 234, total: 281, max: 672
2016-05-17 21:29:22,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:22,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:22,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:22,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:22,289 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:22,294 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:22,295 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:22,295 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:22,296 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:22,296 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:22,297 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 627572587E81491DACEB3829A469FF60, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:22,297 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, ABD6007A8D6E4EBA94C0A316630DFA00, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:22,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:22,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 627572587E81491DACEB3829A469FF60
2016-05-17 21:29:22,299 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:22,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 627572587E81491DACEB3829A469FF60
2016-05-17 21:29:22,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 627572587E81491DACEB3829A469FF60 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:22,300 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:22,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:22,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:22,304 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520560702_0001_1_00_000000_0
2016-05-17 21:29:22,306 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:22,314 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1c55252c
2016-05-17 21:29:22,327 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1c55252c
2016-05-17 21:29:22,328 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:22,328 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:22,348 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:22,359 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 627572587E81491DACEB3829A469FF60
2016-05-17 21:29:22,359 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C54BBA524BF645FAAB761FF54DFEE7B3
2016-05-17 21:29:22,360 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C54BBA524BF645FAAB761FF54DFEE7B3
2016-05-17 21:29:22,380 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:22,396 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:22,397 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@252ce07a; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@25457ca7; comparator=cascading.tuple.hadoop.util.TupleComparator@74facb0e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:22,400 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:22,402 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:22,402 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:22,523 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:22,523 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:22,524 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:22,524 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:22,524 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:22,527 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:22,527 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:22,529 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B7362DDF9FA7463399DCB5FA7D16FC24, all 1 inputs ready in: 00:00:00.002
2016-05-17 21:29:22,538 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520560702_0001_1_00_000000_0
2016-05-17 21:29:22,538 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B7362DDF9FA7463399DCB5FA7D16FC24, mem on close (mb), free: 123, total: 281, max: 672
2016-05-17 21:29:22,539 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:22,539 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:22,540 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:22,540 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:22,554 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:22 Completed running task attempt: attempt_1463520560702_0001_1_00_000000_0
2016-05-17 21:29:22,572 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520560702_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:22,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520560702_0001_1_00, eventCount=1
2016-05-17 21:29:22,575 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520562529, Process_Duration=9, Process_End_Time=1463520562538, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:22,575 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:22,578 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:22,584 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:22,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B7362DDF9FA7463399DCB5FA7D16FC24, taskAttemptId=attempt_1463520560702_0001_1_00_000000_0, startTime=1463520562196, finishTime=1463520562573, timeTaken=377, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520562529, Process_Duration=9, Process_End_Time=1463520562538, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:22,585 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520560702_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:22,586 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:22,587 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520560702_0001_00_000001
2016-05-17 21:29:22,588 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520560702_0001_1_00_000000_0
2016-05-17 21:29:22,591 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:22,595 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:TASK_FINISHED]: vertexName=B7362DDF9FA7463399DCB5FA7D16FC24, taskId=task_1463520560702_0001_1_00_000000, startTime=1463520562196, finishTime=1463520562590, timeTaken=394, status=SUCCEEDED, successfulAttemptID=attempt_1463520560702_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520562529, Process_Duration=9, Process_End_Time=1463520562538, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:22,596 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520560702_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:22,598 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520560702_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:22,598 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24] : 1
2016-05-17 21:29:22,598 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24]
2016-05-17 21:29:22,607 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:22,609 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:VERTEX_FINISHED]: vertexName=B7362DDF9FA7463399DCB5FA7D16FC24, vertexId=vertex_1463520560702_0001_1_00, initRequestedTime=1463520561860, initedTime=1463520562057, startRequestedTime=1463520562058, startedTime=1463520562058, finishTime=1463520562598, timeTaken=540, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520562529, Process_Duration=9, Process_End_Time=1463520562538, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520562196, firstTasksToStart=[ task_1463520560702_0001_1_00_000000 ], lastTaskFinishTime=1463520562573, lastTasksToFinish=[ task_1463520560702_0001_1_00_000000 ], minTaskDuration=377, maxTaskDuration=377, avgTaskDuration=377.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520560702_0001_1_00_000000 ], longestDurationTasks=[ task_1463520560702_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:22,610 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:22,617 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:22,617 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520560702_0001_00_000001, stoppedTime=1463520562617, exitStatus=0
2016-05-17 21:29:22,619 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520560702_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:22,619 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520560702_0001_00_000001: 31 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:22,619 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520560702_0001_00_000001, Exiting
2016-05-17 21:29:22,619 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520560702_0001_00_000001
2016-05-17 21:29:22,619 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520560702_0001_00_000001
2016-05-17 21:29:22,621 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520560702_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:22,622 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00] attempt: attempt_1463520560702_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:22,622 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: ABD6007A8D6E4EBA94C0A316630DFA00
2016-05-17 21:29:22,623 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00]
2016-05-17 21:29:22,624 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520560702_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:22,626 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520560702_0001_00_000001 completed successfully
2016-05-17 21:29:22,626 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:22,628 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520560702_0001_00_000001
2016-05-17 21:29:22,628 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:22,629 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:TASK_STARTED]: vertexName=ABD6007A8D6E4EBA94C0A316630DFA00, taskId=task_1463520560702_0001_1_01_000000, scheduledTime=1463520562628, launchTime=1463520562628
2016-05-17 21:29:22,629 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520560702_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:22,630 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520560702_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:22,630 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00]
2016-05-17 21:29:22,631 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C01539B622544ACC99C0AE01B7C83D2D/ACB2310AB3344A7890BC22B6A7F7F508] every/(1/1) ...blyplatform/-1_'field'_-1, VertexName: ABD6007A8D6E4EBA94C0A316630DFA00, VertexParallelism: 1, TaskAttemptID:attempt_1463520560702_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B7362DDF9FA7463399DCB5FA7D16FC24, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D74B710FE1F747DE84A9A6829102573C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:22,631 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520560702_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:22,632 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:22,633 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520560702_0001_00_000002
2016-05-17 21:29:22,633 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520560702_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:22,636 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520560702_0001_00_000002
2016-05-17 21:29:22,637 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:22,639 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520560702_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:22,639 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520560702_0001_00_000002, launchTime=1463520562639
2016-05-17 21:29:22,639 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:22,641 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520560702_0001_00_000002
2016-05-17 21:29:22,642 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520560702_0001_1_01_000000_0] to container: [container_1463520560702_0001_00_000002]
2016-05-17 21:29:22,642 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520560702_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:22,642 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520560702_0001_00_000002 given task: attempt_1463520560702_0001_1_01_000000_0
2016-05-17 21:29:22,643 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520560702_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520560702_0001_1_01_000000_0
2016-05-17 21:29:22 Starting to run new task attempt: attempt_1463520560702_0001_1_01_000000_0
2016-05-17 21:29:22,644 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520560702_0001_1_01_000000_0
2016-05-17 21:29:22,644 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:22,644 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:22,645 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:22,645 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520560702_0001_1_01_000000_0] started. Is using containerId: [container_1463520560702_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:22,645 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:22,645 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:22,646 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:22,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C01539B622544ACC99C0AE01B7C83D2D/ACB2310AB3344A7890BC22B6A7F7F508] every/(1/1) ...blyplatform/-1_'field'_-1, VertexName: ABD6007A8D6E4EBA94C0A316630DFA00, VertexParallelism: 1, TaskAttemptID:attempt_1463520560702_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B7362DDF9FA7463399DCB5FA7D16FC24, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D74B710FE1F747DE84A9A6829102573C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:22,647 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:22,645 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ABD6007A8D6E4EBA94C0A316630DFA00, taskAttemptId=attempt_1463520560702_0001_1_01_000000_0, startTime=1463520562643, containerId=container_1463520560702_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520560702_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:22,645 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:22,648 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520560702_0001_1_01_000000_0
2016-05-17 21:29:22,650 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:22,650 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520560702_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:22,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:22,652 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520560702_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:22,653 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520560702_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:22,653 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B7362DDF9FA7463399DCB5FA7D16FC24, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:22,653 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:22,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:22,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D74B710FE1F747DE84A9A6829102573C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:22,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:22,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B7362DDF9FA7463399DCB5FA7D16FC24
2016-05-17 21:29:22,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D74B710FE1F747DE84A9A6829102573C
2016-05-17 21:29:22,657 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:22,658 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D74B710FE1F747DE84A9A6829102573C
2016-05-17 21:29:22,660 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:22,680 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:22,680 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:22,680 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B7362DDF9FA7463399DCB5FA7D16FC24
2016-05-17 21:29:22,681 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B7362DDF9FA7463399DCB5FA7D16FC24
2016-05-17 21:29:22,687 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ABD6007A8D6E4EBA94C0A316630DFA00, ordinal: 1
2016-05-17 21:29:22,687 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ABD6007A8D6E4EBA94C0A316630DFA00, mem on start (mb), free: 111, total: 281, max: 672
2016-05-17 21:29:22,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:22,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:22,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:22,688 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:22,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D74B710FE1F747DE84A9A6829102573C
2016-05-17 21:29:22,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:22,690 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:22,690 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:22,690 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:22,690 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:22,690 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:22,691 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:22,691 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D74B710FE1F747DE84A9A6829102573C, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:22,691 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B7362DDF9FA7463399DCB5FA7D16FC24, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:22,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:22,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B7362DDF9FA7463399DCB5FA7D16FC24 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:22,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:22,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:22,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B7362DDF9FA7463399DCB5FA7D16FC24
2016-05-17 21:29:22,701 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:22,701 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:22,705 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B7362DDF9FA7463399DCB5FA7D16FC24 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:22,707 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:22,707 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:22,712 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:22,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B7362DDF9FA7463399DCB5FA7D16FC24
2016-05-17 21:29:22,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:22,717 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520560702_0001_1_01_000000_0
2016-05-17 21:29:22,717 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520560702_0001_1_00_000000_0_10003, runDuration: 304694, ]
2016-05-17 21:29:22,722 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:22,734 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520560702_0001_1_00_000000_0_10003] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520562734, TimeTaken=2, Rate=0.09 MB/s
2016-05-17 21:29:22,734 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B7362DDF9FA7463399DCB5FA7D16FC24
2016-05-17 21:29:22,735 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:22,736 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B7362DDF9FA7463399DCB5FA7D16FC24] #1 in 5ms
2016-05-17 21:29:22,736 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B7362DDF9FA7463399DCB5FA7D16FC24] #1, status:false, isInterrupted:false
2016-05-17 21:29:22,737 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:22,739 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:22,744 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:22,745 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:22,749 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C54BBA524BF645FAAB761FF54DFEE7B3
2016-05-17 21:29:22,749 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_-1"], id: D74B710FE1F747DE84A9A6829102573C
2016-05-17 21:29:22,750 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_-1"]
2016-05-17 21:29:22,751 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C54BBA524BF645FAAB761FF54DFEE7B3, for 1 inputs
2016-05-17 21:29:22,754 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:22,759 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:22,759 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B7362DDF9FA7463399DCB5FA7D16FC24
2016-05-17 21:29:22,759 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ABD6007A8D6E4EBA94C0A316630DFA00, all 1 inputs ready in: 00:00:00.007
2016-05-17 21:29:22,760 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:22,764 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:22,776 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@7bfe307e; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@6d21dfca
2016-05-17 21:29:22,785 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:22,785 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:22,786 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520560702_0001_1_01_000000_0
2016-05-17 21:29:22,786 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520560702_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:22,787 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520560702_0001_1_01_000000_0
2016-05-17 21:29:22,787 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ABD6007A8D6E4EBA94C0A316630DFA00, mem on close (mb), free: 104, total: 281, max: 672
2016-05-17 21:29:22,788 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B7362DDF9FA7463399DCB5FA7D16FC24
2016-05-17 21:29:22,788 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:22,790 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:22,790 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:22,790 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:22,790 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:22,790 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:22 Completed running task attempt: attempt_1463520560702_0001_1_01_000000_0
2016-05-17 21:29:22,792 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520560702_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:22,794 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:22,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=34, MERGE_PHASE_TIME=58, FIRST_EVENT_RECEIVED=30, LAST_EVENT_RECEIVED=30][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520562760, Process_Duration=27, Process_End_Time=1463520562787, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:22,798 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ABD6007A8D6E4EBA94C0A316630DFA00, taskAttemptId=attempt_1463520560702_0001_1_01_000000_0, startTime=1463520562643, finishTime=1463520562794, timeTaken=151, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=34, MERGE_PHASE_TIME=58, FIRST_EVENT_RECEIVED=30, LAST_EVENT_RECEIVED=30, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520562760, Process_Duration=27, Process_End_Time=1463520562787, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:22,799 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:22,799 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520560702_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:22,799 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:22,800 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:22,800 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520560702_0001_1_01_000000_0
2016-05-17 21:29:22,800 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520560702_0001_00_000002
2016-05-17 21:29:22,801 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:22,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:TASK_FINISHED]: vertexName=ABD6007A8D6E4EBA94C0A316630DFA00, taskId=task_1463520560702_0001_1_01_000000, startTime=1463520562643, finishTime=1463520562800, timeTaken=157, status=SUCCEEDED, successfulAttemptID=attempt_1463520560702_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=34, MERGE_PHASE_TIME=58, FIRST_EVENT_RECEIVED=30, LAST_EVENT_RECEIVED=30, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520562760, Process_Duration=27, Process_End_Time=1463520562787, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:22,802 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520560702_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:22,802 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00] : 1
2016-05-17 21:29:22,802 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00]
2016-05-17 21:29:22,806 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:22,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:VERTEX_FINISHED]: vertexName=ABD6007A8D6E4EBA94C0A316630DFA00, vertexId=vertex_1463520560702_0001_1_01, initRequestedTime=1463520561892, initedTime=1463520562040, startRequestedTime=1463520562101, startedTime=1463520562101, finishTime=1463520562802, timeTaken=701, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=34, MERGE_PHASE_TIME=58, FIRST_EVENT_RECEIVED=30, LAST_EVENT_RECEIVED=30, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520562760, Process_Duration=27, Process_End_Time=1463520562787, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520562643, firstTasksToStart=[ task_1463520560702_0001_1_01_000000 ], lastTaskFinishTime=1463520562794, lastTasksToFinish=[ task_1463520560702_0001_1_01_000000 ], minTaskDuration=151, maxTaskDuration=151, avgTaskDuration=151.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520560702_0001_1_01_000000 ], longestDurationTasks=[ task_1463520560702_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:22,808 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:22,809 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520560702_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:22,809 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520560702_0001_00_000002, stoppedTime=1463520562809, exitStatus=0
2016-05-17 21:29:22,809 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:22,809 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520560702_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:22,811 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:22,812 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520560702_0001_1
2016-05-17 21:29:22,812 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520560702_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:22,813 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520560702_0001_00_000002: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:22,813 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520560702_0001_00_000002, Exiting
2016-05-17 21:29:22,813 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520560702_0001_00_000002
2016-05-17 21:29:22,813 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520560702_0001_00_000002
2016-05-17 21:29:22,817 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520560702_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:22,817 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520560702_0001_00_000002 completed successfully
2016-05-17 21:29:22,820 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520560702_0001_1
2016-05-17 21:29:22,820 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D74B710FE1F747DE84A9A6829102573C for vertex: vertex_1463520560702_0001_1_01 [ABD6007A8D6E4EBA94C0A316630DFA00]
2016-05-17 21:29:22,830 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520560702_0001_1_00 [B7362DDF9FA7463399DCB5FA7D16FC24]
2016-05-17 21:29:22,836 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520560702_0001_1, queueSize=0
2016-05-17 21:29:22,836 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:22,837 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520560702_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520560702_0001_1, startTime=1463520561854, finishTime=1463520562830, timeTaken=976, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=1901, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=34, MERGE_PHASE_TIME=58, FIRST_EVENT_RECEIVED=30, LAST_EVENT_RECEIVED=30, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041125289, Process_Duration=36, Process_End_Time=2927041125325, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:22,839 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520560702_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:22,839 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520560702_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:22,839 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520560702_0001_00_000002
2016-05-17 21:29:22,839 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520560702_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:22 Completed Dag: dag_1463520560702_0001_1
2016-05-17 21:29:22,841 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520560702_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:22,841 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520560702_0001_1_post
2016-05-17 21:29:22,841 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:22,842 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:22,850 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/-1_'field'_-1, applicationId=application_1463520560702_0001
2016-05-17 21:29:22,854 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:22,854 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:22,854 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:22,854 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:22,854 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:22,855 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:22,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:22,858 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:22,858 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:22,859 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:22,859 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:22,860 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:22,860 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:22,866 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C01539B622544ACC99C0AE01B7C83D2D/ACB2310AB3344A7890BC22B6A7F7F508/.tez/application_1463520560702_0001
2016-05-17 21:29:22,868 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_-1/_temporary
2016-05-17 21:29:22,869 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:02.370
2016-05-17 21:29:22,973 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_-1
2016-05-17 21:29:23,006 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:23,006 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:23,077 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.034
2016-05-17 21:29:23,102 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.059
2016-05-17 21:29:23,103 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:23,103 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:23,104 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:23,123 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:23,138 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_'field'
2016-05-17 21:29:23,240 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:23,241 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:23,241 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_'field'"]
2016-05-17 21:29:23,242 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:23,242 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:23,242 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:23,245 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...atform/-1_'field'_'field'
2016-05-17 21:29:23,248 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:23,267 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:23,273 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:23,273 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:23,275 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:23,287 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2CB50BDA67634B35B37CA98D99BDAAB0/EDB4363326DF43AE914C9F8AACC34DBC/.tez/application_1463520563273_0001 doesn't exist and is created
2016-05-17 21:29:23,311 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:23,311 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:23,312 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2CB50BDA67634B35B37CA98D99BDAAB0/EDB4363326DF43AE914C9F8AACC34DBC/.tez/application_1463520563273_0001
2016-05-17 21:29:23,318 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520563273_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:23,341 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:23,342 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:23,344 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:23,345 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:23,345 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2CB50BDA67634B35B37CA98D99BDAAB0/EDB4363326DF43AE914C9F8AACC34DBC/.tez/application_1463520563273_0001/localmode-log-dir
2016-05-17 21:29:23,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2CB50BDA67634B35B37CA98D99BDAAB0/EDB4363326DF43AE914C9F8AACC34DBC/.tez/application_1463520563273_0001/localmode-log-dir/history.txt.appattempt_1463520563273_0001_000000, maxErrors=10
2016-05-17 21:29:23,346 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:23,347 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520563273_0001_000000, appSubmitTime=1463520563317, launchTime=1463520563317
2016-05-17 21:29:23,352 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:23,354 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:23,356 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:23,363 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:23,364 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:51737
2016-05-17 21:29:23,367 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:23,372 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520563273_0001_000000, startTime=1463520563372
2016-05-17 21:29:23,372 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:23,372 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:23,412 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:23,412 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:23,412 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_'field', applicationId=application_1463520563273_0001, dagName=[2CB50BDA67634B35B37CA98D99BDAAB0/EDB4363326DF43AE914C9F8AACC34DBC] every/(1/1) ...atform/-1_'field'_'field'
2016-05-17 21:29:23,443 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2CB50BDA67634B35B37CA98D99BDAAB0/EDB4363326DF43AE914C9F8AACC34DBC] every/(1/1) ...atform/-1_'field'_'field'
2016-05-17 21:29:23,448 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520563273_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2CB50BDA67634B35B37CA98D99BDAAB0/EDB4363326DF43AE914C9F8AACC34DBC/.tez/application_1463520563273_0001/localmode-log-dir/dag_1463520563273_0001_1.dot
2016-05-17 21:29:23,449 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2CB50BDA67634B35B37CA98D99BDAAB0/EDB4363326DF43AE914C9F8AACC34DBC/.tez/application_1463520563273_0001/localmode-log-dir/dag_1463520563273_0001_1-tez-dag.pb.txt
2016-05-17 21:29:23,478 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520563273_0001_1
2016-05-17 21:29:23,478 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:23,478 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2CB50BDA67634B35B37CA98D99BDAAB0/EDB4363326DF43AE914C9F8AACC34DBC] every/(1/1) ...atform/-1_'field'_'field'
2016-05-17 21:29:23 Running Dag: dag_1463520563273_0001_1
2016-05-17 21:29:23,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520563273_0001_1, submitTime=1463520563443
2016-05-17 21:29:23,492 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:23,493 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E94E86B732464B92BCDB5DC36A206E08
2016-05-17 21:29:23,493 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:23,493 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:23,494 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520563273_0001_1, initTime=1463520563492
2016-05-17 21:29:23,494 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520563273_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:23,494 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:DAG_STARTED]: dagID=dag_1463520563273_0001_1, startTime=1463520563494
2016-05-17 21:29:23,495 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:23,495 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:23,495 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520563273_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:23,495 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_'field', applicationId=application_1463520563273_0001, dagName=[2CB50BDA67634B35B37CA98D99BDAAB0/EDB4363326DF43AE914C9F8AACC34DBC] every/(1/1) ...atform/-1_'field'_'field'
2016-05-17 21:29:23,495 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E85CAAC470B74CCABF195794EA08D5C7 : {8254D52E979142F2AA1C4688BDE5F67A={InputName=8254D52E979142F2AA1C4688BDE5F67A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:23,496 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 8254D52E979142F2AA1C4688BDE5F67A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:23,496 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7]
2016-05-17 21:29:23,496 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7]
2016-05-17 21:29:23,496 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7]
2016-05-17 21:29:23,496 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7]. Starting root input initializers: 1
2016-05-17 21:29:23,497 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:23,497 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08]
2016-05-17 21:29:23,498 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 8254D52E979142F2AA1C4688BDE5F67A on vertex vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7]
2016-05-17 21:29:23,506 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:23,506 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08]
2016-05-17 21:29:23,508 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08]
2016-05-17 21:29:23,508 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08]
2016-05-17 21:29:23,509 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2E307C2A8FD84D99B50E4A7CB88AFC37, vertexId=vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:23,510 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2E307C2A8FD84D99B50E4A7CB88AFC37, vertexId=vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08]
2016-05-17 21:29:23,517 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:23,517 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 8254D52E979142F2AA1C4688BDE5F67A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:23,520 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:23,520 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:23,521 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2E307C2A8FD84D99B50E4A7CB88AFC37, vertexName=E94E86B732464B92BCDB5DC36A206E08, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:23,521 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520563273_0001, with dag id: dag_1463520563273_0001_1
2016-05-17 21:29:23,522 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2E307C2A8FD84D99B50E4A7CB88AFC37, vertexId=vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08]
2016-05-17 21:29:23,525 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:23,527 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:23,527 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 8254D52E979142F2AA1C4688BDE5F67A on vertex vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7]
2016-05-17 21:29:23,527 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E94E86B732464B92BCDB5DC36A206E08, vertexId=vertex_1463520563273_0001_1_01, initRequestedTime=1463520563497, initedTime=1463520563527, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:23,528 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:23,528 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:23,529 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {8254D52E979142F2AA1C4688BDE5F67A=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:23,529 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7] parallelism set to 1
2016-05-17 21:29:23,529 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:23,531 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520563273_0001_1_00, eventCount=1
2016-05-17 21:29:23,532 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E85CAAC470B74CCABF195794EA08D5C7, vertexId=vertex_1463520563273_0001_1_00, initRequestedTime=1463520563495, initedTime=1463520563531, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:23,532 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7] with distanceFromRoot: 0
2016-05-17 21:29:23,532 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:23,533 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:23,533 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E85CAAC470B74CCABF195794EA08D5C7
2016-05-17 21:29:23,533 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7]
2016-05-17 21:29:23,535 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:23,535 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520563273_0001_1_00, startRequestedTime=1463520563533, startedTime=1463520563533
2016-05-17 21:29:23,535 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:23,537 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:23,537 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:TASK_STARTED]: vertexName=E85CAAC470B74CCABF195794EA08D5C7, taskId=task_1463520563273_0001_1_00_000000, scheduledTime=1463520563536, launchTime=1463520563536
2016-05-17 21:29:23,537 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520563273_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:23,537 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520563273_0001_1_00 for vertex: vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08] numStartedSources: 1 numSources: 1
2016-05-17 21:29:23,538 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08] with distanceFromRoot: 1
2016-05-17 21:29:23,539 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7]
2016-05-17 21:29:23,540 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E85CAAC470B74CCABF195794EA08D5C7 in vertex: E94E86B732464B92BCDB5DC36A206E08
2016-05-17 21:29:23,542 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E94E86B732464B92BCDB5DC36A206E08 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:23,544 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:23,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520563273_0001_1_01, startRequestedTime=1463520563539, startedTime=1463520563539
2016-05-17 21:29:23,545 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:23,547 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2CB50BDA67634B35B37CA98D99BDAAB0/EDB4363326DF43AE914C9F8AACC34DBC] every/(1/1) ...atform/-1_'field'_'field', VertexName: E85CAAC470B74CCABF195794EA08D5C7, VertexParallelism: 1, TaskAttemptID:attempt_1463520563273_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8254D52E979142F2AA1C4688BDE5F67A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E94E86B732464B92BCDB5DC36A206E08, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:23,547 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520563273_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:23,547 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:23,548 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:23,550 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:23,550 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:23,552 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520563273_0001_00_000001
2016-05-17 21:29:23,552 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520563273_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:23,555 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520563273_0001_00_000001
2016-05-17 21:29:23,555 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:23,556 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:23,556 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520563273_0001_00_000001, launchTime=1463520563556
2016-05-17 21:29:23,557 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520563273_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:23,557 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520563273_0001_00_000001
2016-05-17 21:29:23,558 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520563273_0001_1_00_000000_0] to container: [container_1463520563273_0001_00_000001]
2016-05-17 21:29:23,558 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520563273_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:23,558 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520563273_0001_00_000001 given task: attempt_1463520563273_0001_1_00_000000_0
2016-05-17 21:29:23,559 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520563273_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520563273_0001_1_00_000000_0
2016-05-17 21:29:23,559 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520563273_0001_1_00_000000_0] started. Is using containerId: [container_1463520563273_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:23 Starting to run new task attempt: attempt_1463520563273_0001_1_00_000000_0
2016-05-17 21:29:23,560 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520563273_0001_1_00_000000_0
2016-05-17 21:29:23,560 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:23,561 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:23,561 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:23,561 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:23,562 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:23,562 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:23,563 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2CB50BDA67634B35B37CA98D99BDAAB0/EDB4363326DF43AE914C9F8AACC34DBC] every/(1/1) ...atform/-1_'field'_'field', VertexName: E85CAAC470B74CCABF195794EA08D5C7, VertexParallelism: 1, TaskAttemptID:attempt_1463520563273_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8254D52E979142F2AA1C4688BDE5F67A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E94E86B732464B92BCDB5DC36A206E08, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:23,563 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:23,561 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E85CAAC470B74CCABF195794EA08D5C7, taskAttemptId=attempt_1463520563273_0001_1_00_000000_0, startTime=1463520563559, containerId=container_1463520563273_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520563273_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:23,561 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:23,566 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520563273_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:23,566 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520563273_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:23,567 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520563273_0001_1_00_000000_0
2016-05-17 21:29:23,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:23,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:23,571 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520563273_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:23,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E94E86B732464B92BCDB5DC36A206E08, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:23,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=8254D52E979142F2AA1C4688BDE5F67A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:23,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:23,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 8254D52E979142F2AA1C4688BDE5F67A
2016-05-17 21:29:23,574 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 8254D52E979142F2AA1C4688BDE5F67A
2016-05-17 21:29:23,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:23,574 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:23,574 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E94E86B732464B92BCDB5DC36A206E08
2016-05-17 21:29:23,577 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:23,577 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 8254D52E979142F2AA1C4688BDE5F67A
2016-05-17 21:29:23,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 8254D52E979142F2AA1C4688BDE5F67A
2016-05-17 21:29:23,579 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:23,579 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E94E86B732464B92BCDB5DC36A206E08
2016-05-17 21:29:23,581 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E94E86B732464B92BCDB5DC36A206E08
2016-05-17 21:29:23,582 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:23,594 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E85CAAC470B74CCABF195794EA08D5C7, ordinal: 0
2016-05-17 21:29:23,594 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E85CAAC470B74CCABF195794EA08D5C7, mem on start (mb), free: 222, total: 397, max: 672
2016-05-17 21:29:23,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:23,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:23,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:23,595 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:23,595 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:23,595 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:23,596 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:23,596 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:23,596 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:23,597 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:23,597 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 8254D52E979142F2AA1C4688BDE5F67A, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:23,598 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E94E86B732464B92BCDB5DC36A206E08, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:23,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:23,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 8254D52E979142F2AA1C4688BDE5F67A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:23,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:23,600 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:23,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 8254D52E979142F2AA1C4688BDE5F67A
2016-05-17 21:29:23,600 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:23,601 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 8254D52E979142F2AA1C4688BDE5F67A
2016-05-17 21:29:23,601 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:23,601 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520563273_0001_1_00_000000_0
2016-05-17 21:29:23,602 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:23,602 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@8fcda14
2016-05-17 21:29:23,604 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@8fcda14
2016-05-17 21:29:23,604 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:23,604 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:23,607 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:23,608 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 8254D52E979142F2AA1C4688BDE5F67A
2016-05-17 21:29:23,608 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 8F11957E6A0C483999DD245B9DD8FF58
2016-05-17 21:29:23,608 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 8F11957E6A0C483999DD245B9DD8FF58
2016-05-17 21:29:23,610 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:23,614 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:23,614 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@676e42ea; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@77eb45b2; comparator=cascading.tuple.hadoop.util.TupleComparator@772ce69f; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:23,615 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:23,615 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:23,615 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:23,654 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:23,654 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:23,655 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:23,655 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:23,656 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:23,656 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:23,657 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:23,657 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E85CAAC470B74CCABF195794EA08D5C7, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:23,661 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520563273_0001_1_00_000000_0
2016-05-17 21:29:23,661 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E85CAAC470B74CCABF195794EA08D5C7, mem on close (mb), free: 118, total: 397, max: 672
2016-05-17 21:29:23,662 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:23,662 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:23,662 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:23,663 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:23,670 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:23 Completed running task attempt: attempt_1463520563273_0001_1_00_000000_0
2016-05-17 21:29:23,675 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520563273_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:23,676 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520563273_0001_1_00, eventCount=1
2016-05-17 21:29:23,677 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:23,678 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E85CAAC470B74CCABF195794EA08D5C7, taskAttemptId=attempt_1463520563273_0001_1_00_000000_0, startTime=1463520563559, finishTime=1463520563677, timeTaken=118, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520563657, Process_Duration=4, Process_End_Time=1463520563661, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:23,678 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520563273_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:23,678 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520563273_0001_1_00_000000_0
2016-05-17 21:29:23,678 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:23,679 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:23,680 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:TASK_FINISHED]: vertexName=E85CAAC470B74CCABF195794EA08D5C7, taskId=task_1463520563273_0001_1_00_000000, startTime=1463520563559, finishTime=1463520563679, timeTaken=120, status=SUCCEEDED, successfulAttemptID=attempt_1463520563273_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520563657, Process_Duration=4, Process_End_Time=1463520563661, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:23,681 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520563273_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:23,682 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7] : 1
2016-05-17 21:29:23,682 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7]
2016-05-17 21:29:23,682 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520563657, Process_Duration=4, Process_End_Time=1463520563661, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:23,683 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:23,683 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:23,684 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520563273_0001_00_000001
2016-05-17 21:29:23,685 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:VERTEX_FINISHED]: vertexName=E85CAAC470B74CCABF195794EA08D5C7, vertexId=vertex_1463520563273_0001_1_00, initRequestedTime=1463520563495, initedTime=1463520563531, startRequestedTime=1463520563533, startedTime=1463520563533, finishTime=1463520563682, timeTaken=149, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520563657, Process_Duration=4, Process_End_Time=1463520563661, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520563559, firstTasksToStart=[ task_1463520563273_0001_1_00_000000 ], lastTaskFinishTime=1463520563677, lastTasksToFinish=[ task_1463520563273_0001_1_00_000000 ], minTaskDuration=118, maxTaskDuration=118, avgTaskDuration=118.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520563273_0001_1_00_000000 ], longestDurationTasks=[ task_1463520563273_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:23,685 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:23,685 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:23,686 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520563273_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:23,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520563273_0001_00_000001, stoppedTime=1463520563687, exitStatus=0
2016-05-17 21:29:23,687 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520563273_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:23,687 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08] attempt: attempt_1463520563273_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:23,687 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E94E86B732464B92BCDB5DC36A206E08
2016-05-17 21:29:23,689 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:23,690 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08]
2016-05-17 21:29:23,690 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:23,691 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520563273_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:23,691 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:23,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:TASK_STARTED]: vertexName=E94E86B732464B92BCDB5DC36A206E08, taskId=task_1463520563273_0001_1_01_000000, scheduledTime=1463520563691, launchTime=1463520563691
2016-05-17 21:29:23,691 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520563273_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:23,692 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520563273_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:23,692 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08]
2016-05-17 21:29:23,692 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2CB50BDA67634B35B37CA98D99BDAAB0/EDB4363326DF43AE914C9F8AACC34DBC] every/(1/1) ...atform/-1_'field'_'field', VertexName: E94E86B732464B92BCDB5DC36A206E08, VertexParallelism: 1, TaskAttemptID:attempt_1463520563273_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E85CAAC470B74CCABF195794EA08D5C7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2E307C2A8FD84D99B50E4A7CB88AFC37, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:23,693 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520563273_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:23,693 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:23,694 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520563273_0001_00_000002
2016-05-17 21:29:23,694 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520563273_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:23,695 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520563273_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:23,695 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520563273_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:23,696 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520563273_0001_00_000001, Exiting
2016-05-17 21:29:23,696 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520563273_0001_00_000001
2016-05-17 21:29:23,696 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520563273_0001_00_000001
2016-05-17 21:29:23,699 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520563273_0001_00_000002
2016-05-17 21:29:23,700 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520563273_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:23,701 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:23,703 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520563273_0001_00_000001 completed successfully
2016-05-17 21:29:23,703 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520563273_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:23,704 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520563273_0001_00_000002, launchTime=1463520563703
2016-05-17 21:29:23,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:23,704 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520563273_0001_00_000002
2016-05-17 21:29:23,704 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520563273_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:23,705 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520563273_0001_1_01_000000_0] to container: [container_1463520563273_0001_00_000002]
2016-05-17 21:29:23,705 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520563273_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:23,706 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520563273_0001_00_000002 given task: attempt_1463520563273_0001_1_01_000000_0
2016-05-17 21:29:23,706 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520563273_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520563273_0001_1_01_000000_0
2016-05-17 21:29:23,706 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520563273_0001_1_01_000000_0] started. Is using containerId: [container_1463520563273_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:23 Starting to run new task attempt: attempt_1463520563273_0001_1_01_000000_0
2016-05-17 21:29:23,706 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:23,707 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520563273_0001_1_01_000000_0
2016-05-17 21:29:23,707 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:23,707 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:23,707 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:23,707 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:23,707 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:23,707 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:23,708 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2CB50BDA67634B35B37CA98D99BDAAB0/EDB4363326DF43AE914C9F8AACC34DBC] every/(1/1) ...atform/-1_'field'_'field', VertexName: E94E86B732464B92BCDB5DC36A206E08, VertexParallelism: 1, TaskAttemptID:attempt_1463520563273_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E85CAAC470B74CCABF195794EA08D5C7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2E307C2A8FD84D99B50E4A7CB88AFC37, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:23,708 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:23,708 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E94E86B732464B92BCDB5DC36A206E08, taskAttemptId=attempt_1463520563273_0001_1_01_000000_0, startTime=1463520563706, containerId=container_1463520563273_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520563273_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:23,710 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520563273_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:23,710 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520563273_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:23,712 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520563273_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:23,715 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520563273_0001_1_01_000000_0
2016-05-17 21:29:23,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:23,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:23,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E85CAAC470B74CCABF195794EA08D5C7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:23,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2E307C2A8FD84D99B50E4A7CB88AFC37, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:23,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:23,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E85CAAC470B74CCABF195794EA08D5C7
2016-05-17 21:29:23,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:23,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:23,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2E307C2A8FD84D99B50E4A7CB88AFC37
2016-05-17 21:29:23,719 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:23,720 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2E307C2A8FD84D99B50E4A7CB88AFC37
2016-05-17 21:29:23,723 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:23,723 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:23,723 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E85CAAC470B74CCABF195794EA08D5C7
2016-05-17 21:29:23,724 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E85CAAC470B74CCABF195794EA08D5C7
2016-05-17 21:29:23,729 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:23,746 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E94E86B732464B92BCDB5DC36A206E08, ordinal: 1
2016-05-17 21:29:23,746 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E94E86B732464B92BCDB5DC36A206E08, mem on start (mb), free: 109, total: 397, max: 672
2016-05-17 21:29:23,747 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:23,748 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:23,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:23,749 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:23,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2E307C2A8FD84D99B50E4A7CB88AFC37
2016-05-17 21:29:23,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:23,750 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:23,751 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:23,751 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:23,751 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:23,752 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:23,753 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:23,753 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2E307C2A8FD84D99B50E4A7CB88AFC37, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:23,754 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E85CAAC470B74CCABF195794EA08D5C7, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:23,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:23,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E85CAAC470B74CCABF195794EA08D5C7
2016-05-17 21:29:23,756 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E85CAAC470B74CCABF195794EA08D5C7 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:23,759 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:23,759 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:23,760 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E85CAAC470B74CCABF195794EA08D5C7 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:23,760 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:23,760 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:23,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:23,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:23,761 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:23,763 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E85CAAC470B74CCABF195794EA08D5C7
2016-05-17 21:29:23,764 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:23,767 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520563273_0001_1_01_000000_0
2016-05-17 21:29:23,767 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520563273_0001_1_00_000000_0_10009, runDuration: 98578, ]
2016-05-17 21:29:23,770 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520563273_0001_1_00_000000_0_10009] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520563769, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:23,770 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E85CAAC470B74CCABF195794EA08D5C7
2016-05-17 21:29:23,770 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:23,770 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [E85CAAC470B74CCABF195794EA08D5C7] #2 in 2ms
2016-05-17 21:29:23,771 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E85CAAC470B74CCABF195794EA08D5C7] #2, status:false, isInterrupted:false
2016-05-17 21:29:23,771 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:23,772 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:23,773 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 8F11957E6A0C483999DD245B9DD8FF58
2016-05-17 21:29:23,773 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_'field'"], id: 2E307C2A8FD84D99B50E4A7CB88AFC37
2016-05-17 21:29:23,773 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:23,774 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_'field'"]
2016-05-17 21:29:23,774 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:23,774 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 8F11957E6A0C483999DD245B9DD8FF58, for 1 inputs
2016-05-17 21:29:23,774 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:23,774 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:23,775 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:23,775 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E85CAAC470B74CCABF195794EA08D5C7
2016-05-17 21:29:23,775 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:23,776 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E94E86B732464B92BCDB5DC36A206E08, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:29:23,781 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:23,783 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@770ed526; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@6ecbeeda
2016-05-17 21:29:23,788 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:23,788 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:23,789 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520563273_0001_1_01_000000_0
2016-05-17 21:29:23,790 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520563273_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:23,791 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520563273_0001_1_01_000000_0
2016-05-17 21:29:23,791 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E94E86B732464B92BCDB5DC36A206E08, mem on close (mb), free: 104, total: 397, max: 672
2016-05-17 21:29:23,791 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E85CAAC470B74CCABF195794EA08D5C7
2016-05-17 21:29:23,792 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:23,794 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:23,794 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:23,794 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:23,794 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:23,795 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:23 Completed running task attempt: attempt_1463520563273_0001_1_01_000000_0
2016-05-17 21:29:23,797 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520563273_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:23,798 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:23,799 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E94E86B732464B92BCDB5DC36A206E08, taskAttemptId=attempt_1463520563273_0001_1_01_000000_0, startTime=1463520563706, finishTime=1463520563798, timeTaken=92, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=294, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520563776, Process_Duration=15, Process_End_Time=1463520563791, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:23,800 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520563273_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:23,800 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:23,800 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520563273_0001_1_01_000000_0
2016-05-17 21:29:23,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:TASK_FINISHED]: vertexName=E94E86B732464B92BCDB5DC36A206E08, taskId=task_1463520563273_0001_1_01_000000, startTime=1463520563706, finishTime=1463520563800, timeTaken=94, status=SUCCEEDED, successfulAttemptID=attempt_1463520563273_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=294, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520563776, Process_Duration=15, Process_End_Time=1463520563791, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:23,802 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520563273_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:23,803 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08] : 1
2016-05-17 21:29:23,803 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08]
2016-05-17 21:29:23,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=294, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520563776, Process_Duration=15, Process_End_Time=1463520563791, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:23,804 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:23,805 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:23,805 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:23,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:VERTEX_FINISHED]: vertexName=E94E86B732464B92BCDB5DC36A206E08, vertexId=vertex_1463520563273_0001_1_01, initRequestedTime=1463520563497, initedTime=1463520563527, startRequestedTime=1463520563539, startedTime=1463520563539, finishTime=1463520563803, timeTaken=264, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=294, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520563776, Process_Duration=15, Process_End_Time=1463520563791, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520563706, firstTasksToStart=[ task_1463520563273_0001_1_01_000000 ], lastTaskFinishTime=1463520563798, lastTasksToFinish=[ task_1463520563273_0001_1_01_000000 ], minTaskDuration=92, maxTaskDuration=92, avgTaskDuration=92.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520563273_0001_1_01_000000 ], longestDurationTasks=[ task_1463520563273_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:23,808 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:23,809 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520563273_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:23,809 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520563273_0001_00_000002, stoppedTime=1463520563809, exitStatus=0
2016-05-17 21:29:23,809 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520563273_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:23,810 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:23,807 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520563273_0001_00_000002
2016-05-17 21:29:23,813 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520563273_0001_1
2016-05-17 21:29:23,814 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:23,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520563273_0001_1
2016-05-17 21:29:23,815 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2E307C2A8FD84D99B50E4A7CB88AFC37 for vertex: vertex_1463520563273_0001_1_01 [E94E86B732464B92BCDB5DC36A206E08]
2016-05-17 21:29:23,814 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520563273_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:23,816 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520563273_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:23,818 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:23,819 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520563273_0001_00_000002, Exiting
2016-05-17 21:29:23,819 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520563273_0001_00_000002
2016-05-17 21:29:23,819 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520563273_0001_00_000002
2016-05-17 21:29:23,822 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520563273_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:23,822 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520563273_0001_00_000002 completed successfully
2016-05-17 21:29:23,830 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520563273_0001_1_00 [E85CAAC470B74CCABF195794EA08D5C7]
2016-05-17 21:29:23,832 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520563273_0001_1, queueSize=0
2016-05-17 21:29:23,833 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:23,835 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520563273_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520563273_0001_1, startTime=1463520563494, finishTime=1463520563830, timeTaken=336, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=1543, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=833617920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041127433, Process_Duration=19, Process_End_Time=2927041127452, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:23,835 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520563273_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:23,835 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520563273_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:23,836 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520563273_0001_00_000002
2016-05-17 21:29:23,837 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520563273_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:23 Completed Dag: dag_1463520563273_0001_1
2016-05-17 21:29:23,838 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520563273_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:23,838 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520563273_0001_1_post
2016-05-17 21:29:23,839 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:23,839 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:23,845 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_'field'_'field', applicationId=application_1463520563273_0001
2016-05-17 21:29:23,848 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:23,848 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:23,848 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:23,848 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:23,848 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:23,850 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:23,850 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:23,850 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:23,851 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:23,851 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:23,851 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:23,852 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:23,853 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:23,860 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2CB50BDA67634B35B37CA98D99BDAAB0/EDB4363326DF43AE914C9F8AACC34DBC/.tez/application_1463520563273_0001
2016-05-17 21:29:23,862 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_'field'/_temporary
2016-05-17 21:29:23,862 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.621
2016-05-17 21:29:23,984 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_'field'
2016-05-17 21:29:24,020 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:24,021 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:24,092 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.032
2016-05-17 21:29:24,113 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.053
2016-05-17 21:29:24,114 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:24,114 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:24,115 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:24,140 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:24,150 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_RESULTS
2016-05-17 21:29:24,223 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:24,224 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:24,225 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_RESULTS"]
2016-05-17 21:29:24,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:24,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:24,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:24,231 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...atform/-1_'field'_RESULTS
2016-05-17 21:29:24,232 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:24,255 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:24,263 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:24,264 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:24,264 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:24,275 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FB670458ED4C400EB1CF0AFDCB0696BE/BF5F987018FE419E99CC36C89124CF30/.tez/application_1463520564263_0001 doesn't exist and is created
2016-05-17 21:29:24,319 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:24,319 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:24,320 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FB670458ED4C400EB1CF0AFDCB0696BE/BF5F987018FE419E99CC36C89124CF30/.tez/application_1463520564263_0001
2016-05-17 21:29:24,325 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520564263_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:24,338 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:24,339 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:24,341 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:24,341 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:24,341 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FB670458ED4C400EB1CF0AFDCB0696BE/BF5F987018FE419E99CC36C89124CF30/.tez/application_1463520564263_0001/localmode-log-dir
2016-05-17 21:29:24,342 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FB670458ED4C400EB1CF0AFDCB0696BE/BF5F987018FE419E99CC36C89124CF30/.tez/application_1463520564263_0001/localmode-log-dir/history.txt.appattempt_1463520564263_0001_000000, maxErrors=10
2016-05-17 21:29:24,342 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:24,342 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520564263_0001_000000, appSubmitTime=1463520564324, launchTime=1463520564324
2016-05-17 21:29:24,350 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:24,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:24,354 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:24,354 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:24,358 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49562
2016-05-17 21:29:24,359 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:24,362 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520564263_0001_000000, startTime=1463520564362
2016-05-17 21:29:24,362 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:24,362 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:24,420 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:24,420 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:24,420 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_RESULTS, applicationId=application_1463520564263_0001, dagName=[FB670458ED4C400EB1CF0AFDCB0696BE/BF5F987018FE419E99CC36C89124CF30] every/(1/1) ...atform/-1_'field'_RESULTS
2016-05-17 21:29:24,443 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [FB670458ED4C400EB1CF0AFDCB0696BE/BF5F987018FE419E99CC36C89124CF30] every/(1/1) ...atform/-1_'field'_RESULTS
2016-05-17 21:29:24,445 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520564263_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FB670458ED4C400EB1CF0AFDCB0696BE/BF5F987018FE419E99CC36C89124CF30/.tez/application_1463520564263_0001/localmode-log-dir/dag_1463520564263_0001_1.dot
2016-05-17 21:29:24,447 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FB670458ED4C400EB1CF0AFDCB0696BE/BF5F987018FE419E99CC36C89124CF30/.tez/application_1463520564263_0001/localmode-log-dir/dag_1463520564263_0001_1-tez-dag.pb.txt
2016-05-17 21:29:24,475 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520564263_0001_1
2016-05-17 21:29:24,475 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:24,476 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [FB670458ED4C400EB1CF0AFDCB0696BE/BF5F987018FE419E99CC36C89124CF30] every/(1/1) ...atform/-1_'field'_RESULTS
2016-05-17 21:29:24 Running Dag: dag_1463520564263_0001_1
2016-05-17 21:29:24,490 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:24,490 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520564263_0001_1, submitTime=1463520564443
2016-05-17 21:29:24,491 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D72816A882924BADB8A35B4B103C98E4
2016-05-17 21:29:24,491 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:24,491 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:24,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520564263_0001_1, initTime=1463520564490
2016-05-17 21:29:24,493 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520564263_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:24,493 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:DAG_STARTED]: dagID=dag_1463520564263_0001_1, startTime=1463520564493
2016-05-17 21:29:24,493 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:24,494 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520564263_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:24,494 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_RESULTS, applicationId=application_1463520564263_0001, dagName=[FB670458ED4C400EB1CF0AFDCB0696BE/BF5F987018FE419E99CC36C89124CF30] every/(1/1) ...atform/-1_'field'_RESULTS
2016-05-17 21:29:24,494 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C09A07BCB7234419901DB56EB38F8889 : {D04908B6E7CB42CC85C406D96CD88A7B={InputName=D04908B6E7CB42CC85C406D96CD88A7B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:24,494 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: D04908B6E7CB42CC85C406D96CD88A7B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:24,494 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889]
2016-05-17 21:29:24,494 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889]
2016-05-17 21:29:24,495 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889]
2016-05-17 21:29:24,495 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889]. Starting root input initializers: 1
2016-05-17 21:29:24,495 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:24,495 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:24,496 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: D04908B6E7CB42CC85C406D96CD88A7B on vertex vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889]
2016-05-17 21:29:24,497 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:24,497 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4]
2016-05-17 21:29:24,503 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:24,503 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4]
2016-05-17 21:29:24,503 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input D04908B6E7CB42CC85C406D96CD88A7B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:24,504 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4]
2016-05-17 21:29:24,504 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4]
2016-05-17 21:29:24,505 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=37AE83939A1F45A5833982640DA6614A, vertexId=vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:24,505 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=37AE83939A1F45A5833982640DA6614A, vertexId=vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4]
2016-05-17 21:29:24,506 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:24,507 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:24,508 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=37AE83939A1F45A5833982640DA6614A, vertexName=D72816A882924BADB8A35B4B103C98E4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:24,509 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=37AE83939A1F45A5833982640DA6614A, vertexId=vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4]
2016-05-17 21:29:24,511 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:24,512 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D72816A882924BADB8A35B4B103C98E4, vertexId=vertex_1463520564263_0001_1_01, initRequestedTime=1463520564496, initedTime=1463520564511, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:24,513 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:24,514 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:24,515 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: D04908B6E7CB42CC85C406D96CD88A7B on vertex vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889]
2016-05-17 21:29:24,515 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:24,516 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {D04908B6E7CB42CC85C406D96CD88A7B=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:24,516 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889] parallelism set to 1
2016-05-17 21:29:24,517 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:24,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520564263_0001_1_00, eventCount=1
2016-05-17 21:29:24,518 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C09A07BCB7234419901DB56EB38F8889, vertexId=vertex_1463520564263_0001_1_00, initRequestedTime=1463520564494, initedTime=1463520564518, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:24,519 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889] with distanceFromRoot: 0
2016-05-17 21:29:24,519 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:24,518 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:24,520 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C09A07BCB7234419901DB56EB38F8889
2016-05-17 21:29:24,520 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889]
2016-05-17 21:29:24,521 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520564263_0001_1_00, startRequestedTime=1463520564520, startedTime=1463520564520
2016-05-17 21:29:24,522 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:24,522 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:24,523 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:TASK_STARTED]: vertexName=C09A07BCB7234419901DB56EB38F8889, taskId=task_1463520564263_0001_1_00_000000, scheduledTime=1463520564523, launchTime=1463520564523
2016-05-17 21:29:24,523 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:24,524 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520564263_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:24,525 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520564263_0001_1_00 for vertex: vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4] numStartedSources: 1 numSources: 1
2016-05-17 21:29:24,525 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4] with distanceFromRoot: 1
2016-05-17 21:29:24,525 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889]
2016-05-17 21:29:24,526 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C09A07BCB7234419901DB56EB38F8889 in vertex: D72816A882924BADB8A35B4B103C98E4
2016-05-17 21:29:24,527 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D72816A882924BADB8A35B4B103C98E4 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:24,527 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520564263_0001_1_01, startRequestedTime=1463520564526, startedTime=1463520564526
2016-05-17 21:29:24,527 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520564263_0001, with dag id: dag_1463520564263_0001_1
2016-05-17 21:29:24,527 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:24,528 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:24,529 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FB670458ED4C400EB1CF0AFDCB0696BE/BF5F987018FE419E99CC36C89124CF30] every/(1/1) ...atform/-1_'field'_RESULTS, VertexName: C09A07BCB7234419901DB56EB38F8889, VertexParallelism: 1, TaskAttemptID:attempt_1463520564263_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D04908B6E7CB42CC85C406D96CD88A7B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D72816A882924BADB8A35B4B103C98E4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:24,530 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520564263_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:24,530 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:24,532 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:24,534 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:24,535 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:24,536 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520564263_0001_00_000001
2016-05-17 21:29:24,536 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520564263_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:24,539 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520564263_0001_00_000001
2016-05-17 21:29:24,539 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:24,541 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520564263_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:24,541 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:24,541 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520564263_0001_00_000001, launchTime=1463520564541
2016-05-17 21:29:24,542 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520564263_0001_00_000001
2016-05-17 21:29:24,542 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520564263_0001_1_00_000000_0] to container: [container_1463520564263_0001_00_000001]
2016-05-17 21:29:24,543 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520564263_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:24,543 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520564263_0001_00_000001 given task: attempt_1463520564263_0001_1_00_000000_0
2016-05-17 21:29:24,543 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520564263_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520564263_0001_1_00_000000_0
2016-05-17 21:29:24,543 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520564263_0001_1_00_000000_0] started. Is using containerId: [container_1463520564263_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:24,544 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C09A07BCB7234419901DB56EB38F8889, taskAttemptId=attempt_1463520564263_0001_1_00_000000_0, startTime=1463520564543, containerId=container_1463520564263_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520564263_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:24,544 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520564263_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:24,544 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:24,544 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520564263_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:24 Starting to run new task attempt: attempt_1463520564263_0001_1_00_000000_0
2016-05-17 21:29:24,545 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520564263_0001_1_00_000000_0
2016-05-17 21:29:24,545 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:24,545 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:24,545 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:24,545 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:24,545 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:24,545 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:24,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FB670458ED4C400EB1CF0AFDCB0696BE/BF5F987018FE419E99CC36C89124CF30] every/(1/1) ...atform/-1_'field'_RESULTS, VertexName: C09A07BCB7234419901DB56EB38F8889, VertexParallelism: 1, TaskAttemptID:attempt_1463520564263_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D04908B6E7CB42CC85C406D96CD88A7B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D72816A882924BADB8A35B4B103C98E4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:24,546 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:24,547 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520564263_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:24,548 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520564263_0001_1_00_000000_0
2016-05-17 21:29:24,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:24,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:24,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D04908B6E7CB42CC85C406D96CD88A7B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:24,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:24,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D72816A882924BADB8A35B4B103C98E4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:24,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:24,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D72816A882924BADB8A35B4B103C98E4
2016-05-17 21:29:24,553 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:24,553 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:24,554 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D72816A882924BADB8A35B4B103C98E4
2016-05-17 21:29:24,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D72816A882924BADB8A35B4B103C98E4
2016-05-17 21:29:24,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:24,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D04908B6E7CB42CC85C406D96CD88A7B
2016-05-17 21:29:24,556 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: D04908B6E7CB42CC85C406D96CD88A7B
2016-05-17 21:29:24,559 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:24,560 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: D04908B6E7CB42CC85C406D96CD88A7B
2016-05-17 21:29:24,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D04908B6E7CB42CC85C406D96CD88A7B
2016-05-17 21:29:24,567 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C09A07BCB7234419901DB56EB38F8889, ordinal: 0
2016-05-17 21:29:24,567 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C09A07BCB7234419901DB56EB38F8889, mem on start (mb), free: 230, total: 397, max: 672
2016-05-17 21:29:24,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:24,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:24,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:24,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:24,568 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:24,568 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:24,568 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:24,569 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:24,569 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:24,569 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:24,569 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D72816A882924BADB8A35B4B103C98E4, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:24,569 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D04908B6E7CB42CC85C406D96CD88A7B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:24,569 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:24,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D04908B6E7CB42CC85C406D96CD88A7B
2016-05-17 21:29:24,570 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:24,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D04908B6E7CB42CC85C406D96CD88A7B
2016-05-17 21:29:24,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D04908B6E7CB42CC85C406D96CD88A7B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:24,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:24,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:24,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:24,571 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520564263_0001_1_00_000000_0
2016-05-17 21:29:24,572 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:24,573 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6dfd4209
2016-05-17 21:29:24,576 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:24,576 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6dfd4209
2016-05-17 21:29:24,577 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:24,577 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:24,577 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: D04908B6E7CB42CC85C406D96CD88A7B
2016-05-17 21:29:24,580 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 21EAA30D10E34EA6AC924D78665F3A83
2016-05-17 21:29:24,580 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 21EAA30D10E34EA6AC924D78665F3A83
2016-05-17 21:29:24,582 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:24,586 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:24,587 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@44ab73a3; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@5434f58d; comparator=cascading.tuple.hadoop.util.TupleComparator@60e5272b; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:24,588 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:24,588 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:24,589 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:24,602 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:24,603 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:24,603 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:24,604 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:24,604 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:24,605 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:24,605 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:24,606 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C09A07BCB7234419901DB56EB38F8889, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:24,609 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520564263_0001_1_00_000000_0
2016-05-17 21:29:24,609 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C09A07BCB7234419901DB56EB38F8889, mem on close (mb), free: 128, total: 397, max: 672
2016-05-17 21:29:24,609 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:24,610 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:24,610 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:24,610 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:24,615 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:24 Completed running task attempt: attempt_1463520564263_0001_1_00_000000_0
2016-05-17 21:29:24,620 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520564263_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:24,620 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520564263_0001_1_00, eventCount=1
2016-05-17 21:29:24,622 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C09A07BCB7234419901DB56EB38F8889, taskAttemptId=attempt_1463520564263_0001_1_00_000000_0, startTime=1463520564543, finishTime=1463520564621, timeTaken=78, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520564606, Process_Duration=3, Process_End_Time=1463520564609, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:24,622 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520564263_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:24,623 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520564606, Process_Duration=3, Process_End_Time=1463520564609, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:24,622 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:24,624 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:24,623 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:24,623 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520564263_0001_1_00_000000_0
2016-05-17 21:29:24,625 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:24,626 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520564263_0001_00_000001
2016-05-17 21:29:24,626 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:24,626 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:TASK_FINISHED]: vertexName=C09A07BCB7234419901DB56EB38F8889, taskId=task_1463520564263_0001_1_00_000000, startTime=1463520564543, finishTime=1463520564625, timeTaken=82, status=SUCCEEDED, successfulAttemptID=attempt_1463520564263_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520564606, Process_Duration=3, Process_End_Time=1463520564609, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:24,627 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520564263_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:24,627 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889] : 1
2016-05-17 21:29:24,627 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889]
2016-05-17 21:29:24,629 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:24,630 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:VERTEX_FINISHED]: vertexName=C09A07BCB7234419901DB56EB38F8889, vertexId=vertex_1463520564263_0001_1_00, initRequestedTime=1463520564494, initedTime=1463520564518, startRequestedTime=1463520564520, startedTime=1463520564520, finishTime=1463520564627, timeTaken=107, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520564606, Process_Duration=3, Process_End_Time=1463520564609, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520564543, firstTasksToStart=[ task_1463520564263_0001_1_00_000000 ], lastTaskFinishTime=1463520564621, lastTasksToFinish=[ task_1463520564263_0001_1_00_000000 ], minTaskDuration=78, maxTaskDuration=78, avgTaskDuration=78.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520564263_0001_1_00_000000 ], longestDurationTasks=[ task_1463520564263_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:24,630 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:24,630 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520564263_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:24,630 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520564263_0001_00_000001, stoppedTime=1463520564630, exitStatus=0
2016-05-17 21:29:24,630 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520564263_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:24,630 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:24,631 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4] attempt: attempt_1463520564263_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:24,631 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D72816A882924BADB8A35B4B103C98E4
2016-05-17 21:29:24,632 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4]
2016-05-17 21:29:24,632 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:24,632 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520564263_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:24,633 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:24,633 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:TASK_STARTED]: vertexName=D72816A882924BADB8A35B4B103C98E4, taskId=task_1463520564263_0001_1_01_000000, scheduledTime=1463520564633, launchTime=1463520564633
2016-05-17 21:29:24,633 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520564263_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:24,634 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520564263_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:24,634 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4]
2016-05-17 21:29:24,635 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FB670458ED4C400EB1CF0AFDCB0696BE/BF5F987018FE419E99CC36C89124CF30] every/(1/1) ...atform/-1_'field'_RESULTS, VertexName: D72816A882924BADB8A35B4B103C98E4, VertexParallelism: 1, TaskAttemptID:attempt_1463520564263_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C09A07BCB7234419901DB56EB38F8889, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=37AE83939A1F45A5833982640DA6614A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:24,635 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520564263_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:24,635 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:24,636 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520564263_0001_00_000002
2016-05-17 21:29:24,636 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520564263_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:24,637 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520564263_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:24,637 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520564263_0001_00_000001, Exiting
2016-05-17 21:29:24,637 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520564263_0001_00_000001
2016-05-17 21:29:24,637 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520564263_0001_00_000001
2016-05-17 21:29:24,636 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520564263_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:24,639 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520564263_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:24,639 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520564263_0001_00_000001 completed successfully
2016-05-17 21:29:24,639 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520564263_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:24,640 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520564263_0001_00_000002
2016-05-17 21:29:24,641 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:24,643 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:24,644 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520564263_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:24,644 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520564263_0001_00_000002, launchTime=1463520564643
2016-05-17 21:29:24,645 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520564263_0001_00_000002
2016-05-17 21:29:24,646 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520564263_0001_1_01_000000_0] to container: [container_1463520564263_0001_00_000002]
2016-05-17 21:29:24,646 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520564263_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:24,646 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520564263_0001_00_000002 given task: attempt_1463520564263_0001_1_01_000000_0
2016-05-17 21:29:24,646 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520564263_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520564263_0001_1_01_000000_0
2016-05-17 21:29:24,647 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520564263_0001_1_01_000000_0] started. Is using containerId: [container_1463520564263_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:24 Starting to run new task attempt: attempt_1463520564263_0001_1_01_000000_0
2016-05-17 21:29:24,647 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520564263_0001_1_01_000000_0
2016-05-17 21:29:24,647 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:24,647 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:24,647 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:24,647 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:24,647 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D72816A882924BADB8A35B4B103C98E4, taskAttemptId=attempt_1463520564263_0001_1_01_000000_0, startTime=1463520564646, containerId=container_1463520564263_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520564263_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:24,647 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:24,648 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520564263_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:24,648 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:24,648 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520564263_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:24,648 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:24,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FB670458ED4C400EB1CF0AFDCB0696BE/BF5F987018FE419E99CC36C89124CF30] every/(1/1) ...atform/-1_'field'_RESULTS, VertexName: D72816A882924BADB8A35B4B103C98E4, VertexParallelism: 1, TaskAttemptID:attempt_1463520564263_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C09A07BCB7234419901DB56EB38F8889, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=37AE83939A1F45A5833982640DA6614A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:24,649 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:24,649 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520564263_0001_1_01_000000_0
2016-05-17 21:29:24,650 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:24,650 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:24,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C09A07BCB7234419901DB56EB38F8889, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:24,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=37AE83939A1F45A5833982640DA6614A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:24,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:24,652 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:24,652 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 37AE83939A1F45A5833982640DA6614A
2016-05-17 21:29:24,652 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:24,652 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 37AE83939A1F45A5833982640DA6614A
2016-05-17 21:29:24,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:24,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C09A07BCB7234419901DB56EB38F8889
2016-05-17 21:29:24,657 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:24,658 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:24,658 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:24,658 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C09A07BCB7234419901DB56EB38F8889
2016-05-17 21:29:24,658 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520564263_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:24,662 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C09A07BCB7234419901DB56EB38F8889
2016-05-17 21:29:24,669 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D72816A882924BADB8A35B4B103C98E4, ordinal: 1
2016-05-17 21:29:24,669 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D72816A882924BADB8A35B4B103C98E4, mem on start (mb), free: 120, total: 397, max: 672
2016-05-17 21:29:24,669 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:24,669 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:24,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:24,672 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:24,672 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 37AE83939A1F45A5833982640DA6614A
2016-05-17 21:29:24,672 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:24,672 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:24,672 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:24,673 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:24,673 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:24,673 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:24,673 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:24,674 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 37AE83939A1F45A5833982640DA6614A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:24,674 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C09A07BCB7234419901DB56EB38F8889, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:24,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:24,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C09A07BCB7234419901DB56EB38F8889 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:24,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:24,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:24,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C09A07BCB7234419901DB56EB38F8889
2016-05-17 21:29:24,676 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:24,676 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:24,677 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C09A07BCB7234419901DB56EB38F8889 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:24,677 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:24,677 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:24,677 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:24,678 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C09A07BCB7234419901DB56EB38F8889
2016-05-17 21:29:24,678 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:24,679 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520564263_0001_1_01_000000_0
2016-05-17 21:29:24,683 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:24,683 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520564263_0001_1_00_000000_0_10014, runDuration: 66406, ]
2016-05-17 21:29:24,684 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 21EAA30D10E34EA6AC924D78665F3A83
2016-05-17 21:29:24,684 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_RESULTS"], id: 37AE83939A1F45A5833982640DA6614A
2016-05-17 21:29:24,685 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_RESULTS"]
2016-05-17 21:29:24,685 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 21EAA30D10E34EA6AC924D78665F3A83, for 1 inputs
2016-05-17 21:29:24,685 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520564263_0001_1_00_000000_0_10014] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520564685, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:24,686 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C09A07BCB7234419901DB56EB38F8889
2016-05-17 21:29:24,686 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:24,686 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [C09A07BCB7234419901DB56EB38F8889] #3 in 2ms
2016-05-17 21:29:24,687 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C09A07BCB7234419901DB56EB38F8889] #3, status:false, isInterrupted:false
2016-05-17 21:29:24,687 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:24,689 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:24,689 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:24,689 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:24,689 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:24,690 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:24,690 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C09A07BCB7234419901DB56EB38F8889
2016-05-17 21:29:24,690 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D72816A882924BADB8A35B4B103C98E4, all 1 inputs ready in: 00:00:00.005
2016-05-17 21:29:24,691 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:24,694 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:24,696 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@483e6be3; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@7a1d2922
2016-05-17 21:29:24,700 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:24,700 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:24,701 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520564263_0001_1_01_000000_0
2016-05-17 21:29:24,701 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520564263_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:24,702 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520564263_0001_1_01_000000_0
2016-05-17 21:29:24,702 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D72816A882924BADB8A35B4B103C98E4, mem on close (mb), free: 115, total: 397, max: 672
2016-05-17 21:29:24,702 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C09A07BCB7234419901DB56EB38F8889
2016-05-17 21:29:24,703 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:24,705 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:24,705 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:24,705 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:24,705 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:24,705 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:24 Completed running task attempt: attempt_1463520564263_0001_1_01_000000_0
2016-05-17 21:29:24,706 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520564263_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:24,708 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:24,709 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D72816A882924BADB8A35B4B103C98E4, taskAttemptId=attempt_1463520564263_0001_1_01_000000_0, startTime=1463520564646, finishTime=1463520564708, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520564691, Process_Duration=11, Process_End_Time=1463520564702, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:24,709 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520564263_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:24,709 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520564263_0001_1_01_000000_0
2016-05-17 21:29:24,710 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:24,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520564691, Process_Duration=11, Process_End_Time=1463520564702, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:24,711 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:24,711 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:TASK_FINISHED]: vertexName=D72816A882924BADB8A35B4B103C98E4, taskId=task_1463520564263_0001_1_01_000000, startTime=1463520564646, finishTime=1463520564709, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1463520564263_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520564691, Process_Duration=11, Process_End_Time=1463520564702, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:24,711 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:24,713 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520564263_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:24,713 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4] : 1
2016-05-17 21:29:24,713 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4]
2016-05-17 21:29:24,712 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:24,715 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520564263_0001_00_000002
2016-05-17 21:29:24,715 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:VERTEX_FINISHED]: vertexName=D72816A882924BADB8A35B4B103C98E4, vertexId=vertex_1463520564263_0001_1_01, initRequestedTime=1463520564496, initedTime=1463520564511, startRequestedTime=1463520564526, startedTime=1463520564526, finishTime=1463520564714, timeTaken=188, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=416808960, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520564691, Process_Duration=11, Process_End_Time=1463520564702, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520564646, firstTasksToStart=[ task_1463520564263_0001_1_01_000000 ], lastTaskFinishTime=1463520564708, lastTasksToFinish=[ task_1463520564263_0001_1_01_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520564263_0001_1_01_000000 ], longestDurationTasks=[ task_1463520564263_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:24,716 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:24,716 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520564263_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:24,716 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520564263_0001_00_000002, stoppedTime=1463520564716, exitStatus=0
2016-05-17 21:29:24,717 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520564263_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:24,717 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:24,717 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520564263_0001_1
2016-05-17 21:29:24,717 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520564263_0001_1
2016-05-17 21:29:24,717 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 37AE83939A1F45A5833982640DA6614A for vertex: vertex_1463520564263_0001_1_01 [D72816A882924BADB8A35B4B103C98E4]
2016-05-17 21:29:24,716 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:24,719 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:24,725 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520564263_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:24,725 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520564263_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:24,725 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520564263_0001_00_000002, Exiting
2016-05-17 21:29:24,726 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520564263_0001_00_000002
2016-05-17 21:29:24,726 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520564263_0001_00_000002
2016-05-17 21:29:24,727 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520564263_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:24,727 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520564263_0001_00_000002 completed successfully
2016-05-17 21:29:24,728 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520564263_0001_1_00 [C09A07BCB7234419901DB56EB38F8889]
2016-05-17 21:29:24,730 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520564263_0001_1, queueSize=0
2016-05-17 21:29:24,731 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:24,732 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520564263_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520564263_0001_1, startTime=1463520564493, finishTime=1463520564728, timeTaken=235, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=865, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=833617920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041129297, Process_Duration=14, Process_End_Time=2927041129311, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:24,732 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520564263_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:24,732 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520564263_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:24,733 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520564263_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:24,733 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520564263_0001_00_000002
2016-05-17 21:29:24 Completed Dag: dag_1463520564263_0001_1
2016-05-17 21:29:24,734 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520564263_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:24,734 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520564263_0001_1_post
2016-05-17 21:29:24,734 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:24,734 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:24,742 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_'field'_RESULTS, applicationId=application_1463520564263_0001
2016-05-17 21:29:24,745 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:24,745 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:24,746 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:24,746 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:24,747 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:24,747 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:24,748 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:24,748 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:24,749 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:24,749 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:24,749 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:24,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:24,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:24,756 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FB670458ED4C400EB1CF0AFDCB0696BE/BF5F987018FE419E99CC36C89124CF30/.tez/application_1463520564263_0001
2016-05-17 21:29:24,757 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_RESULTS/_temporary
2016-05-17 21:29:24,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.533
2016-05-17 21:29:24,869 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_RESULTS
2016-05-17 21:29:24,894 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:24,895 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:24,945 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-17 21:29:24,965 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.041
2016-05-17 21:29:24,966 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:24,966 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:24,967 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:24,986 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:24,993 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_ALL
2016-05-17 21:29:25,051 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:25,052 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:25,052 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_ALL"]
2016-05-17 21:29:25,053 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:25,053 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:25,053 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:25,054 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...lyplatform/-1_'field'_ALL
2016-05-17 21:29:25,054 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:25,068 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:25,072 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:25,073 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:25,073 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:25,082 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/067A4E1CD1064FCEA91EF23FF369CA0E/A038AC1E0E8D471AACE53DF2C9683180/.tez/application_1463520565073_0001 doesn't exist and is created
2016-05-17 21:29:25,100 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:25,100 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:25,101 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/067A4E1CD1064FCEA91EF23FF369CA0E/A038AC1E0E8D471AACE53DF2C9683180/.tez/application_1463520565073_0001
2016-05-17 21:29:25,105 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520565073_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:25,116 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:25,117 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:25,119 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:25,119 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:25,120 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/067A4E1CD1064FCEA91EF23FF369CA0E/A038AC1E0E8D471AACE53DF2C9683180/.tez/application_1463520565073_0001/localmode-log-dir
2016-05-17 21:29:25,120 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/067A4E1CD1064FCEA91EF23FF369CA0E/A038AC1E0E8D471AACE53DF2C9683180/.tez/application_1463520565073_0001/localmode-log-dir/history.txt.appattempt_1463520565073_0001_000000, maxErrors=10
2016-05-17 21:29:25,120 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:25,121 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520565073_0001_000000, appSubmitTime=1463520565104, launchTime=1463520565104
2016-05-17 21:29:25,130 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53256
2016-05-17 21:29:25,130 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:25,133 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:25,135 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:25,135 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:25,136 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:25,139 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520565073_0001_000000, startTime=1463520565139
2016-05-17 21:29:25,139 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:25,139 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:25,200 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:25,201 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:25,201 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/-1_'field'_ALL, applicationId=application_1463520565073_0001, dagName=[067A4E1CD1064FCEA91EF23FF369CA0E/A038AC1E0E8D471AACE53DF2C9683180] every/(1/1) ...lyplatform/-1_'field'_ALL
2016-05-17 21:29:25,246 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [067A4E1CD1064FCEA91EF23FF369CA0E/A038AC1E0E8D471AACE53DF2C9683180] every/(1/1) ...lyplatform/-1_'field'_ALL
2016-05-17 21:29:25,248 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520565073_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/067A4E1CD1064FCEA91EF23FF369CA0E/A038AC1E0E8D471AACE53DF2C9683180/.tez/application_1463520565073_0001/localmode-log-dir/dag_1463520565073_0001_1.dot
2016-05-17 21:29:25,250 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/067A4E1CD1064FCEA91EF23FF369CA0E/A038AC1E0E8D471AACE53DF2C9683180/.tez/application_1463520565073_0001/localmode-log-dir/dag_1463520565073_0001_1-tez-dag.pb.txt
2016-05-17 21:29:25,282 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520565073_0001_1
2016-05-17 21:29:25,282 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:25,282 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [067A4E1CD1064FCEA91EF23FF369CA0E/A038AC1E0E8D471AACE53DF2C9683180] every/(1/1) ...lyplatform/-1_'field'_ALL
2016-05-17 21:29:25 Running Dag: dag_1463520565073_0001_1
2016-05-17 21:29:25,296 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:25,296 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520565073_0001_1, submitTime=1463520565246
2016-05-17 21:29:25,297 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A39772DF256546EB8FE5EE0757B98982
2016-05-17 21:29:25,298 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:25,298 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:25,299 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520565073_0001_1, initTime=1463520565297
2016-05-17 21:29:25,299 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520565073_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:25,300 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:DAG_STARTED]: dagID=dag_1463520565073_0001_1, startTime=1463520565299
2016-05-17 21:29:25,300 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:25,300 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520565073_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:25,300 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C35FB30EDF2740F9A5F3BF1267926FEB : {CF7F19E247904C71B4D529F4FDDB3EE1={InputName=CF7F19E247904C71B4D529F4FDDB3EE1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:25,300 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: CF7F19E247904C71B4D529F4FDDB3EE1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:25,300 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB]
2016-05-17 21:29:25,301 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB]
2016-05-17 21:29:25,301 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB]
2016-05-17 21:29:25,301 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:25,301 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB]. Starting root input initializers: 1
2016-05-17 21:29:25,302 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:25,302 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: CF7F19E247904C71B4D529F4FDDB3EE1 on vertex vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB]
2016-05-17 21:29:25,302 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982]
2016-05-17 21:29:25,303 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/-1_'field'_ALL, applicationId=application_1463520565073_0001, dagName=[067A4E1CD1064FCEA91EF23FF369CA0E/A038AC1E0E8D471AACE53DF2C9683180] every/(1/1) ...lyplatform/-1_'field'_ALL
2016-05-17 21:29:25,306 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:25,306 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982]
2016-05-17 21:29:25,306 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982]
2016-05-17 21:29:25,306 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982]
2016-05-17 21:29:25,306 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=C68EB37C19A04362BAAB8315D58811E7, vertexId=vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:25,307 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=C68EB37C19A04362BAAB8315D58811E7, vertexId=vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982]
2016-05-17 21:29:25,307 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:25,307 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input CF7F19E247904C71B4D529F4FDDB3EE1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:25,308 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:25,308 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:25,309 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=C68EB37C19A04362BAAB8315D58811E7, vertexName=A39772DF256546EB8FE5EE0757B98982, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:25,310 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=C68EB37C19A04362BAAB8315D58811E7, vertexId=vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982]
2016-05-17 21:29:25,311 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:25,311 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: CF7F19E247904C71B4D529F4FDDB3EE1 on vertex vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB]
2016-05-17 21:29:25,312 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:25,312 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A39772DF256546EB8FE5EE0757B98982, vertexId=vertex_1463520565073_0001_1_01, initRequestedTime=1463520565302, initedTime=1463520565312, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:25,312 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:25,312 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:25,313 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {CF7F19E247904C71B4D529F4FDDB3EE1=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:25,313 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB] parallelism set to 1
2016-05-17 21:29:25,313 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:25,313 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520565073_0001_1_00, eventCount=1
2016-05-17 21:29:25,314 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:25,314 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C35FB30EDF2740F9A5F3BF1267926FEB, vertexId=vertex_1463520565073_0001_1_00, initRequestedTime=1463520565300, initedTime=1463520565314, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:25,315 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB] with distanceFromRoot: 0
2016-05-17 21:29:25,315 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:25,315 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C35FB30EDF2740F9A5F3BF1267926FEB
2016-05-17 21:29:25,315 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB]
2016-05-17 21:29:25,316 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520565073_0001_1_00, startRequestedTime=1463520565315, startedTime=1463520565315
2016-05-17 21:29:25,316 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:25,316 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:25,317 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:25,317 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:TASK_STARTED]: vertexName=C35FB30EDF2740F9A5F3BF1267926FEB, taskId=task_1463520565073_0001_1_00_000000, scheduledTime=1463520565316, launchTime=1463520565316
2016-05-17 21:29:25,317 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520565073_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:25,318 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520565073_0001_1_00 for vertex: vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982] numStartedSources: 1 numSources: 1
2016-05-17 21:29:25,318 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982] with distanceFromRoot: 1
2016-05-17 21:29:25,318 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB]
2016-05-17 21:29:25,319 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C35FB30EDF2740F9A5F3BF1267926FEB in vertex: A39772DF256546EB8FE5EE0757B98982
2016-05-17 21:29:25,319 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A39772DF256546EB8FE5EE0757B98982 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:25,319 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:25,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520565073_0001_1_01, startRequestedTime=1463520565319, startedTime=1463520565319
2016-05-17 21:29:25,320 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:25,320 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [067A4E1CD1064FCEA91EF23FF369CA0E/A038AC1E0E8D471AACE53DF2C9683180] every/(1/1) ...lyplatform/-1_'field'_ALL, VertexName: C35FB30EDF2740F9A5F3BF1267926FEB, VertexParallelism: 1, TaskAttemptID:attempt_1463520565073_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CF7F19E247904C71B4D529F4FDDB3EE1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A39772DF256546EB8FE5EE0757B98982, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:25,321 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520565073_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:25,321 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:25,322 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:25,323 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:25,323 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:25,324 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520565073_0001_00_000001
2016-05-17 21:29:25,324 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520565073_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:25,325 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520565073_0001_00_000001
2016-05-17 21:29:25,325 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:25,326 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520565073_0001, with dag id: dag_1463520565073_0001_1
2016-05-17 21:29:25,326 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520565073_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:25,326 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:25,328 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520565073_0001_00_000001, launchTime=1463520565326
2016-05-17 21:29:25,332 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520565073_0001_00_000001
2016-05-17 21:29:25,334 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520565073_0001_1_00_000000_0] to container: [container_1463520565073_0001_00_000001]
2016-05-17 21:29:25,334 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520565073_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:25,335 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520565073_0001_00_000001 given task: attempt_1463520565073_0001_1_00_000000_0
2016-05-17 21:29:25,336 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520565073_0001_1_00_000000_0] started. Is using containerId: [container_1463520565073_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:25,337 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:25,337 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C35FB30EDF2740F9A5F3BF1267926FEB, taskAttemptId=attempt_1463520565073_0001_1_00_000000_0, startTime=1463520565335, containerId=container_1463520565073_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520565073_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:25,337 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520565073_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:25,337 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520565073_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:25,338 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520565073_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520565073_0001_1_00_000000_0
2016-05-17 21:29:25 Starting to run new task attempt: attempt_1463520565073_0001_1_00_000000_0
2016-05-17 21:29:25,338 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520565073_0001_1_00_000000_0
2016-05-17 21:29:25,339 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:25,339 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:25,339 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:25,339 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:25,341 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:25,341 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:25,341 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [067A4E1CD1064FCEA91EF23FF369CA0E/A038AC1E0E8D471AACE53DF2C9683180] every/(1/1) ...lyplatform/-1_'field'_ALL, VertexName: C35FB30EDF2740F9A5F3BF1267926FEB, VertexParallelism: 1, TaskAttemptID:attempt_1463520565073_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CF7F19E247904C71B4D529F4FDDB3EE1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A39772DF256546EB8FE5EE0757B98982, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:25,342 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:25,344 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520565073_0001_1_00_000000_0
2016-05-17 21:29:25,344 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520565073_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:25,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:25,345 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:25,345 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CF7F19E247904C71B4D529F4FDDB3EE1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:25,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A39772DF256546EB8FE5EE0757B98982, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:25,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:25,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A39772DF256546EB8FE5EE0757B98982
2016-05-17 21:29:25,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:25,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:25,347 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CF7F19E247904C71B4D529F4FDDB3EE1
2016-05-17 21:29:25,347 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: CF7F19E247904C71B4D529F4FDDB3EE1
2016-05-17 21:29:25,348 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:25,349 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A39772DF256546EB8FE5EE0757B98982
2016-05-17 21:29:25,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A39772DF256546EB8FE5EE0757B98982
2016-05-17 21:29:25,350 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:25,350 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: CF7F19E247904C71B4D529F4FDDB3EE1
2016-05-17 21:29:25,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CF7F19E247904C71B4D529F4FDDB3EE1
2016-05-17 21:29:25,351 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:25,359 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C35FB30EDF2740F9A5F3BF1267926FEB, ordinal: 0
2016-05-17 21:29:25,359 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C35FB30EDF2740F9A5F3BF1267926FEB, mem on start (mb), free: 212, total: 373, max: 672
2016-05-17 21:29:25,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:25,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:25,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:25,360 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:25,360 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:25,360 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:25,361 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:25,361 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:25,361 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:25,362 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:25,362 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CF7F19E247904C71B4D529F4FDDB3EE1, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:25,362 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A39772DF256546EB8FE5EE0757B98982, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:25,362 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:25,363 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CF7F19E247904C71B4D529F4FDDB3EE1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:25,363 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:25,363 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:25,363 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CF7F19E247904C71B4D529F4FDDB3EE1
2016-05-17 21:29:25,364 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:25,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CF7F19E247904C71B4D529F4FDDB3EE1
2016-05-17 21:29:25,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:25,365 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520565073_0001_1_00_000000_0
2016-05-17 21:29:25,365 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:25,366 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3352b82b
2016-05-17 21:29:25,367 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3352b82b
2016-05-17 21:29:25,368 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:25,368 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:25,370 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:25,371 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: CF7F19E247904C71B4D529F4FDDB3EE1
2016-05-17 21:29:25,371 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: DA7F09D32AE24E88A9181FFEBD56DB9E
2016-05-17 21:29:25,371 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] DA7F09D32AE24E88A9181FFEBD56DB9E
2016-05-17 21:29:25,373 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:25,375 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:25,375 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@727d9aac; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@1435cced; comparator=cascading.tuple.hadoop.util.TupleComparator@5820e5f8; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:25,376 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:25,376 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:25,376 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:25,389 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:25,390 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:25,390 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:25,390 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:25,390 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:25,390 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:25,391 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:25,391 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C35FB30EDF2740F9A5F3BF1267926FEB, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:25,393 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520565073_0001_1_00_000000_0
2016-05-17 21:29:25,394 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C35FB30EDF2740F9A5F3BF1267926FEB, mem on close (mb), free: 111, total: 373, max: 672
2016-05-17 21:29:25,394 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:25,394 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:25,394 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:25,394 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:25,399 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:25 Completed running task attempt: attempt_1463520565073_0001_1_00_000000_0
2016-05-17 21:29:25,403 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520565073_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:25,404 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520565073_0001_1_00, eventCount=1
2016-05-17 21:29:25,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=391643136, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520565391, Process_Duration=2, Process_End_Time=1463520565393, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:25,405 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:25,405 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C35FB30EDF2740F9A5F3BF1267926FEB, taskAttemptId=attempt_1463520565073_0001_1_00_000000_0, startTime=1463520565335, finishTime=1463520565405, timeTaken=70, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=391643136, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520565391, Process_Duration=2, Process_End_Time=1463520565393, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:25,405 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:25,406 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520565073_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:25,406 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:25,406 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:25,406 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520565073_0001_1_00_000000_0
2016-05-17 21:29:25,407 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520565073_0001_00_000001
2016-05-17 21:29:25,407 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:25,408 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:TASK_FINISHED]: vertexName=C35FB30EDF2740F9A5F3BF1267926FEB, taskId=task_1463520565073_0001_1_00_000000, startTime=1463520565335, finishTime=1463520565407, timeTaken=72, status=SUCCEEDED, successfulAttemptID=attempt_1463520565073_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=391643136, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520565391, Process_Duration=2, Process_End_Time=1463520565393, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:25,408 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520565073_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:25,408 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB] : 1
2016-05-17 21:29:25,408 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB]
2016-05-17 21:29:25,410 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:25,411 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:VERTEX_FINISHED]: vertexName=C35FB30EDF2740F9A5F3BF1267926FEB, vertexId=vertex_1463520565073_0001_1_00, initRequestedTime=1463520565300, initedTime=1463520565314, startRequestedTime=1463520565315, startedTime=1463520565315, finishTime=1463520565409, timeTaken=94, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=391643136, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520565391, Process_Duration=2, Process_End_Time=1463520565393, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520565335, firstTasksToStart=[ task_1463520565073_0001_1_00_000000 ], lastTaskFinishTime=1463520565405, lastTasksToFinish=[ task_1463520565073_0001_1_00_000000 ], minTaskDuration=70, maxTaskDuration=70, avgTaskDuration=70.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520565073_0001_1_00_000000 ], longestDurationTasks=[ task_1463520565073_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:25,411 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:25,411 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520565073_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:25,411 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520565073_0001_00_000001, stoppedTime=1463520565411, exitStatus=0
2016-05-17 21:29:25,412 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520565073_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:25,412 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982] attempt: attempt_1463520565073_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:25,412 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:25,412 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A39772DF256546EB8FE5EE0757B98982
2016-05-17 21:29:25,412 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982]
2016-05-17 21:29:25,412 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:25,413 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:25,413 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520565073_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:25,413 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:TASK_STARTED]: vertexName=A39772DF256546EB8FE5EE0757B98982, taskId=task_1463520565073_0001_1_01_000000, scheduledTime=1463520565413, launchTime=1463520565413
2016-05-17 21:29:25,413 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520565073_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:25,414 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982]
2016-05-17 21:29:25,414 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520565073_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:25,414 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [067A4E1CD1064FCEA91EF23FF369CA0E/A038AC1E0E8D471AACE53DF2C9683180] every/(1/1) ...lyplatform/-1_'field'_ALL, VertexName: A39772DF256546EB8FE5EE0757B98982, VertexParallelism: 1, TaskAttemptID:attempt_1463520565073_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C35FB30EDF2740F9A5F3BF1267926FEB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C68EB37C19A04362BAAB8315D58811E7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:25,414 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520565073_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:25,414 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:25,416 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520565073_0001_00_000002
2016-05-17 21:29:25,416 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520565073_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:25,418 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520565073_0001_00_000002
2016-05-17 21:29:25,418 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:25,418 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520565073_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:25,418 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520565073_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:25,418 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520565073_0001_00_000001, Exiting
2016-05-17 21:29:25,418 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520565073_0001_00_000001
2016-05-17 21:29:25,418 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520565073_0001_00_000001
2016-05-17 21:29:25,421 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:25,422 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520565073_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:25,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520565073_0001_00_000002, launchTime=1463520565420
2016-05-17 21:29:25,423 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520565073_0001_00_000002
2016-05-17 21:29:25,423 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520565073_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:25,423 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520565073_0001_00_000001 completed successfully
2016-05-17 21:29:25,423 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520565073_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:25,425 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520565073_0001_1_01_000000_0] to container: [container_1463520565073_0001_00_000002]
2016-05-17 21:29:25,426 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520565073_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:25,426 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520565073_0001_1_01_000000_0] started. Is using containerId: [container_1463520565073_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:25,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:25,426 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520565073_0001_00_000002 given task: attempt_1463520565073_0001_1_01_000000_0
2016-05-17 21:29:25,427 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520565073_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520565073_0001_1_01_000000_0
2016-05-17 21:29:25 Starting to run new task attempt: attempt_1463520565073_0001_1_01_000000_0
2016-05-17 21:29:25,428 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520565073_0001_1_01_000000_0
2016-05-17 21:29:25,429 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:25,430 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:25,430 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:25,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A39772DF256546EB8FE5EE0757B98982, taskAttemptId=attempt_1463520565073_0001_1_01_000000_0, startTime=1463520565426, containerId=container_1463520565073_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520565073_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:25,430 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520565073_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:25,431 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520565073_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:25,430 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:25,432 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:25,432 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:25,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [067A4E1CD1064FCEA91EF23FF369CA0E/A038AC1E0E8D471AACE53DF2C9683180] every/(1/1) ...lyplatform/-1_'field'_ALL, VertexName: A39772DF256546EB8FE5EE0757B98982, VertexParallelism: 1, TaskAttemptID:attempt_1463520565073_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C35FB30EDF2740F9A5F3BF1267926FEB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C68EB37C19A04362BAAB8315D58811E7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:25,433 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:25,433 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520565073_0001_1_01_000000_0
2016-05-17 21:29:25,433 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:25,434 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:25,434 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520565073_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:25,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C35FB30EDF2740F9A5F3BF1267926FEB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:25,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C68EB37C19A04362BAAB8315D58811E7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:25,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:25,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:25,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C35FB30EDF2740F9A5F3BF1267926FEB
2016-05-17 21:29:25,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:25,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C68EB37C19A04362BAAB8315D58811E7
2016-05-17 21:29:25,438 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:25,438 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: C68EB37C19A04362BAAB8315D58811E7
2016-05-17 21:29:25,440 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:25,440 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:25,440 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C35FB30EDF2740F9A5F3BF1267926FEB
2016-05-17 21:29:25,441 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C35FB30EDF2740F9A5F3BF1267926FEB
2016-05-17 21:29:25,440 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:25,451 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:25,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C68EB37C19A04362BAAB8315D58811E7
2016-05-17 21:29:25,454 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A39772DF256546EB8FE5EE0757B98982, ordinal: 1
2016-05-17 21:29:25,454 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A39772DF256546EB8FE5EE0757B98982, mem on start (mb), free: 103, total: 373, max: 672
2016-05-17 21:29:25,454 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:25,454 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:25,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:25,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:25,456 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:25,457 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:25,457 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:25,457 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:25,458 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:25,458 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:25,458 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C68EB37C19A04362BAAB8315D58811E7, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:25,459 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C35FB30EDF2740F9A5F3BF1267926FEB, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:25,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:25,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C35FB30EDF2740F9A5F3BF1267926FEB
2016-05-17 21:29:25,460 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:25,461 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:25,461 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C35FB30EDF2740F9A5F3BF1267926FEB with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:25,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C35FB30EDF2740F9A5F3BF1267926FEB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:25,462 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:25,462 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:25,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:25,463 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:25,462 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:25,464 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C35FB30EDF2740F9A5F3BF1267926FEB
2016-05-17 21:29:25,464 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:25,467 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520565073_0001_1_01_000000_0
2016-05-17 21:29:25,467 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520565073_0001_1_00_000000_0_10021, runDuration: 54962, ]
2016-05-17 21:29:25,471 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:25,472 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: DA7F09D32AE24E88A9181FFEBD56DB9E
2016-05-17 21:29:25,472 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520565073_0001_1_00_000000_0_10021] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520565472, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:25,473 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C35FB30EDF2740F9A5F3BF1267926FEB
2016-05-17 21:29:25,473 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:25,473 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [C35FB30EDF2740F9A5F3BF1267926FEB] #4 in 2ms
2016-05-17 21:29:25,473 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_ALL"], id: C68EB37C19A04362BAAB8315D58811E7
2016-05-17 21:29:25,476 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_ALL"]
2016-05-17 21:29:25,476 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] DA7F09D32AE24E88A9181FFEBD56DB9E, for 1 inputs
2016-05-17 21:29:25,476 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C35FB30EDF2740F9A5F3BF1267926FEB] #4, status:false, isInterrupted:false
2016-05-17 21:29:25,478 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:25,479 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:25,479 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:25,479 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:25,480 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:25,480 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:25,480 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C35FB30EDF2740F9A5F3BF1267926FEB
2016-05-17 21:29:25,481 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:25,481 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A39772DF256546EB8FE5EE0757B98982, all 1 inputs ready in: 00:00:00.004
2016-05-17 21:29:25,483 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:25,485 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@57506a12; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@6063f086
2016-05-17 21:29:25,488 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:25,489 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:25,489 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520565073_0001_1_01_000000_0
2016-05-17 21:29:25,489 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520565073_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:25,490 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520565073_0001_1_01_000000_0
2016-05-17 21:29:25,490 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A39772DF256546EB8FE5EE0757B98982, mem on close (mb), free: 99, total: 373, max: 672
2016-05-17 21:29:25,490 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C35FB30EDF2740F9A5F3BF1267926FEB
2016-05-17 21:29:25,491 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:25,492 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:25,492 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:25,492 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:25,493 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:25,493 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:25 Completed running task attempt: attempt_1463520565073_0001_1_01_000000_0
2016-05-17 21:29:25,494 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520565073_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:25,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=391643136, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520565481, Process_Duration=9, Process_End_Time=1463520565490, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:25,495 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:25,495 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:25,496 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:25,496 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A39772DF256546EB8FE5EE0757B98982, taskAttemptId=attempt_1463520565073_0001_1_01_000000_0, startTime=1463520565426, finishTime=1463520565495, timeTaken=69, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=391643136, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520565481, Process_Duration=9, Process_End_Time=1463520565490, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:25,496 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520565073_0001_00_000002
2016-05-17 21:29:25,496 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520565073_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:25,497 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520565073_0001_1_01_000000_0
2016-05-17 21:29:25,497 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:25,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:TASK_FINISHED]: vertexName=A39772DF256546EB8FE5EE0757B98982, taskId=task_1463520565073_0001_1_01_000000, startTime=1463520565426, finishTime=1463520565497, timeTaken=71, status=SUCCEEDED, successfulAttemptID=attempt_1463520565073_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=391643136, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520565481, Process_Duration=9, Process_End_Time=1463520565490, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:25,498 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520565073_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:25,498 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982] : 1
2016-05-17 21:29:25,498 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982]
2016-05-17 21:29:25,499 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:25,499 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:VERTEX_FINISHED]: vertexName=A39772DF256546EB8FE5EE0757B98982, vertexId=vertex_1463520565073_0001_1_01, initRequestedTime=1463520565302, initedTime=1463520565312, startRequestedTime=1463520565319, startedTime=1463520565319, finishTime=1463520565498, timeTaken=179, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=391643136, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520565481, Process_Duration=9, Process_End_Time=1463520565490, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520565426, firstTasksToStart=[ task_1463520565073_0001_1_01_000000 ], lastTaskFinishTime=1463520565495, lastTasksToFinish=[ task_1463520565073_0001_1_01_000000 ], minTaskDuration=69, maxTaskDuration=69, avgTaskDuration=69.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520565073_0001_1_01_000000 ], longestDurationTasks=[ task_1463520565073_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:25,500 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:25,500 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:25,500 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:25,500 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520565073_0001_1
2016-05-17 21:29:25,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520565073_0001_1
2016-05-17 21:29:25,501 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520565073_0001_1_00 [C35FB30EDF2740F9A5F3BF1267926FEB]
2016-05-17 21:29:25,501 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: C68EB37C19A04362BAAB8315D58811E7 for vertex: vertex_1463520565073_0001_1_01 [A39772DF256546EB8FE5EE0757B98982]
2016-05-17 21:29:25,511 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520565073_0001_1, queueSize=0
2016-05-17 21:29:25,511 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:25,513 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520565073_0001_1, startTime=1463520565299, finishTime=1463520565508, timeTaken=209, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=451, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=783286272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041130872, Process_Duration=11, Process_End_Time=2927041130883, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:25,513 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520565073_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:25,513 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520565073_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:25,514 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520565073_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:25,514 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:25,515 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520565073_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520565073_0001_00_000002, stoppedTime=1463520565514, exitStatus=0
2016-05-17 21:29:25,515 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520565073_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:25 Completed Dag: dag_1463520565073_0001_1
2016-05-17 21:29:25,515 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520565073_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:25,515 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520565073_0001_1_post
2016-05-17 21:29:25,515 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:25,516 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:25,516 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520565073_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:25,516 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520565073_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:25,522 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/-1_'field'_ALL, applicationId=application_1463520565073_0001
2016-05-17 21:29:25,525 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:25,525 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:25,525 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:25,525 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:25,525 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:25,526 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:25,526 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:25,526 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:25,527 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:25,527 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520565073_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:25,527 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:25,527 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520565073_0001_00_000002: 31 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:25,527 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520565073_0001_00_000002, Exiting
2016-05-17 21:29:25,527 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520565073_0001_00_000002
2016-05-17 21:29:25,528 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520565073_0001_00_000002
2016-05-17 21:29:25,527 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:25,529 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520565073_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:25,529 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520565073_0001_00_000002 completed successfully
2016-05-17 21:29:25,529 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:25,530 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:25,529 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520565073_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:25,535 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/067A4E1CD1064FCEA91EF23FF369CA0E/A038AC1E0E8D471AACE53DF2C9683180/.tez/application_1463520565073_0001
2016-05-17 21:29:25,536 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_ALL/_temporary
2016-05-17 21:29:25,536 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.484
2016-05-17 21:29:25,609 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_ALL
2016-05-17 21:29:25,637 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:25,638 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:25,676 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.009
2016-05-17 21:29:25,681 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.015
2016-05-17 21:29:25,682 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:25,682 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:25,682 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_REPLACE
2016-05-17 21:29:25,708 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:25,709 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:25,752 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-17 21:29:25,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.029
2016-05-17 21:29:25,766 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:25,766 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:25,767 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:25,785 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:25,793 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_SWAP
2016-05-17 21:29:26,022 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:26,023 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:26,023 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_SWAP"]
2016-05-17 21:29:26,024 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:26,025 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:26,025 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:26,026 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...yplatform/-1_'field'_SWAP
2016-05-17 21:29:26,026 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:26,040 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:26,044 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:26,045 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:26,045 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:26,054 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A14F8255CD564B9CA3B79372A9B1A6D1/FA73B69024EA4C5AB63E576DDEE9ED98/.tez/application_1463520566045_0001 doesn't exist and is created
2016-05-17 21:29:26,071 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:26,071 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:26,071 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A14F8255CD564B9CA3B79372A9B1A6D1/FA73B69024EA4C5AB63E576DDEE9ED98/.tez/application_1463520566045_0001
2016-05-17 21:29:26,075 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520566045_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:26,086 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:26,087 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:26,088 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:26,089 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:26,089 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A14F8255CD564B9CA3B79372A9B1A6D1/FA73B69024EA4C5AB63E576DDEE9ED98/.tez/application_1463520566045_0001/localmode-log-dir
2016-05-17 21:29:26,089 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A14F8255CD564B9CA3B79372A9B1A6D1/FA73B69024EA4C5AB63E576DDEE9ED98/.tez/application_1463520566045_0001/localmode-log-dir/history.txt.appattempt_1463520566045_0001_000000, maxErrors=10
2016-05-17 21:29:26,089 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:26,090 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520566045_0001_000000, appSubmitTime=1463520566075, launchTime=1463520566075
2016-05-17 21:29:26,094 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:26,095 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:26,097 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:26,097 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:26,102 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:47926
2016-05-17 21:29:26,103 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:26,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520566045_0001_000000, startTime=1463520566105
2016-05-17 21:29:26,105 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:26,106 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:26,172 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:26,172 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:26,172 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_'field'_SWAP, applicationId=application_1463520566045_0001, dagName=[A14F8255CD564B9CA3B79372A9B1A6D1/FA73B69024EA4C5AB63E576DDEE9ED98] every/(1/1) ...yplatform/-1_'field'_SWAP
2016-05-17 21:29:26,190 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A14F8255CD564B9CA3B79372A9B1A6D1/FA73B69024EA4C5AB63E576DDEE9ED98] every/(1/1) ...yplatform/-1_'field'_SWAP
2016-05-17 21:29:26,192 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520566045_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A14F8255CD564B9CA3B79372A9B1A6D1/FA73B69024EA4C5AB63E576DDEE9ED98/.tez/application_1463520566045_0001/localmode-log-dir/dag_1463520566045_0001_1.dot
2016-05-17 21:29:26,193 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A14F8255CD564B9CA3B79372A9B1A6D1/FA73B69024EA4C5AB63E576DDEE9ED98/.tez/application_1463520566045_0001/localmode-log-dir/dag_1463520566045_0001_1-tez-dag.pb.txt
2016-05-17 21:29:26,218 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520566045_0001_1
2016-05-17 21:29:26,218 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:26,218 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A14F8255CD564B9CA3B79372A9B1A6D1/FA73B69024EA4C5AB63E576DDEE9ED98] every/(1/1) ...yplatform/-1_'field'_SWAP
2016-05-17 21:29:26 Running Dag: dag_1463520566045_0001_1
2016-05-17 21:29:26,230 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520566045_0001_1, submitTime=1463520566190
2016-05-17 21:29:26,230 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:26,232 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A242C9CA4C084292A57A6DCF2C13FC39
2016-05-17 21:29:26,232 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:26,233 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520566045_0001_1, initTime=1463520566231
2016-05-17 21:29:26,233 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:26,233 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520566045_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:26,235 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:DAG_STARTED]: dagID=dag_1463520566045_0001_1, startTime=1463520566235
2016-05-17 21:29:26,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:26,235 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:26,235 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520566045_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:26,235 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D8B6B648356F412FB3DE9CBDB12E4538 : {68F05DA95D4F4906877529EC0EA5AF83={InputName=68F05DA95D4F4906877529EC0EA5AF83}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:26,236 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 68F05DA95D4F4906877529EC0EA5AF83, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:26,236 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538]
2016-05-17 21:29:26,236 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538]
2016-05-17 21:29:26,236 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538]
2016-05-17 21:29:26,237 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538]. Starting root input initializers: 1
2016-05-17 21:29:26,237 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:26,237 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39]
2016-05-17 21:29:26,237 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 68F05DA95D4F4906877529EC0EA5AF83 on vertex vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538]
2016-05-17 21:29:26,238 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_'field'_SWAP, applicationId=application_1463520566045_0001, dagName=[A14F8255CD564B9CA3B79372A9B1A6D1/FA73B69024EA4C5AB63E576DDEE9ED98] every/(1/1) ...yplatform/-1_'field'_SWAP
2016-05-17 21:29:26,240 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:26,240 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:26,241 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39]
2016-05-17 21:29:26,241 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 68F05DA95D4F4906877529EC0EA5AF83 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:26,241 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39]
2016-05-17 21:29:26,242 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39]
2016-05-17 21:29:26,242 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=AA0AFEB707134C28B90DB99614C34B77, vertexId=vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:26,242 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=AA0AFEB707134C28B90DB99614C34B77, vertexId=vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39]
2016-05-17 21:29:26,242 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:26,243 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:26,245 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=AA0AFEB707134C28B90DB99614C34B77, vertexName=A242C9CA4C084292A57A6DCF2C13FC39, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:26,245 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=AA0AFEB707134C28B90DB99614C34B77, vertexId=vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39]
2016-05-17 21:29:26,246 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:26,247 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:26,248 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A242C9CA4C084292A57A6DCF2C13FC39, vertexId=vertex_1463520566045_0001_1_01, initRequestedTime=1463520566237, initedTime=1463520566247, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:26,248 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:26,249 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 68F05DA95D4F4906877529EC0EA5AF83 on vertex vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538]
2016-05-17 21:29:26,249 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:26,249 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {68F05DA95D4F4906877529EC0EA5AF83=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:26,250 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538] parallelism set to 1
2016-05-17 21:29:26,250 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:26,251 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520566045_0001_1_00, eventCount=1
2016-05-17 21:29:26,252 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D8B6B648356F412FB3DE9CBDB12E4538, vertexId=vertex_1463520566045_0001_1_00, initRequestedTime=1463520566235, initedTime=1463520566252, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:26,253 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538] with distanceFromRoot: 0
2016-05-17 21:29:26,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:26,253 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:26,254 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D8B6B648356F412FB3DE9CBDB12E4538
2016-05-17 21:29:26,254 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538]
2016-05-17 21:29:26,254 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520566045_0001_1_00, startRequestedTime=1463520566253, startedTime=1463520566253
2016-05-17 21:29:26,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:26,255 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:26,255 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:26,255 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:TASK_STARTED]: vertexName=D8B6B648356F412FB3DE9CBDB12E4538, taskId=task_1463520566045_0001_1_00_000000, scheduledTime=1463520566255, launchTime=1463520566255
2016-05-17 21:29:26,256 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520566045_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:26,256 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520566045_0001_1_00 for vertex: vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39] numStartedSources: 1 numSources: 1
2016-05-17 21:29:26,256 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39] with distanceFromRoot: 1
2016-05-17 21:29:26,257 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538]
2016-05-17 21:29:26,257 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D8B6B648356F412FB3DE9CBDB12E4538 in vertex: A242C9CA4C084292A57A6DCF2C13FC39
2016-05-17 21:29:26,257 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A242C9CA4C084292A57A6DCF2C13FC39 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:26,258 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520566045_0001_1_01, startRequestedTime=1463520566257, startedTime=1463520566257
2016-05-17 21:29:26,258 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:26,258 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:26,259 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A14F8255CD564B9CA3B79372A9B1A6D1/FA73B69024EA4C5AB63E576DDEE9ED98] every/(1/1) ...yplatform/-1_'field'_SWAP, VertexName: D8B6B648356F412FB3DE9CBDB12E4538, VertexParallelism: 1, TaskAttemptID:attempt_1463520566045_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=68F05DA95D4F4906877529EC0EA5AF83, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A242C9CA4C084292A57A6DCF2C13FC39, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:26,259 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520566045_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:26,259 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:26,260 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:26,266 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:26,266 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:26,267 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520566045_0001, with dag id: dag_1463520566045_0001_1
2016-05-17 21:29:26,269 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520566045_0001_00_000001
2016-05-17 21:29:26,269 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566045_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:26,271 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520566045_0001_00_000001
2016-05-17 21:29:26,271 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:26,272 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566045_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:26,274 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:26,274 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520566045_0001_00_000001, launchTime=1463520566272
2016-05-17 21:29:26,276 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520566045_0001_00_000001
2016-05-17 21:29:26,277 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520566045_0001_1_00_000000_0] to container: [container_1463520566045_0001_00_000001]
2016-05-17 21:29:26,277 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566045_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:26,278 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520566045_0001_00_000001 given task: attempt_1463520566045_0001_1_00_000000_0
2016-05-17 21:29:26,278 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520566045_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520566045_0001_1_00_000000_0
2016-05-17 21:29:26,278 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520566045_0001_1_00_000000_0] started. Is using containerId: [container_1463520566045_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:26,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D8B6B648356F412FB3DE9CBDB12E4538, taskAttemptId=attempt_1463520566045_0001_1_00_000000_0, startTime=1463520566278, containerId=container_1463520566045_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520566045_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:26,279 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520566045_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:26,279 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:26,279 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520566045_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:26 Starting to run new task attempt: attempt_1463520566045_0001_1_00_000000_0
2016-05-17 21:29:26,279 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520566045_0001_1_00_000000_0
2016-05-17 21:29:26,279 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:26,279 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:26,280 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:26,280 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:26,280 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:26,280 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:26,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A14F8255CD564B9CA3B79372A9B1A6D1/FA73B69024EA4C5AB63E576DDEE9ED98] every/(1/1) ...yplatform/-1_'field'_SWAP, VertexName: D8B6B648356F412FB3DE9CBDB12E4538, VertexParallelism: 1, TaskAttemptID:attempt_1463520566045_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=68F05DA95D4F4906877529EC0EA5AF83, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A242C9CA4C084292A57A6DCF2C13FC39, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:26,280 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:26,281 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520566045_0001_1_00_000000_0
2016-05-17 21:29:26,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:26,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:26,282 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520566045_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:26,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:26,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A242C9CA4C084292A57A6DCF2C13FC39, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:26,285 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:26,284 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=68F05DA95D4F4906877529EC0EA5AF83, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:26,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:26,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 68F05DA95D4F4906877529EC0EA5AF83
2016-05-17 21:29:26,287 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 68F05DA95D4F4906877529EC0EA5AF83
2016-05-17 21:29:26,285 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:26,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A242C9CA4C084292A57A6DCF2C13FC39
2016-05-17 21:29:26,292 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:26,293 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A242C9CA4C084292A57A6DCF2C13FC39
2016-05-17 21:29:26,294 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:26,295 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 68F05DA95D4F4906877529EC0EA5AF83
2016-05-17 21:29:26,294 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A242C9CA4C084292A57A6DCF2C13FC39
2016-05-17 21:29:26,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 68F05DA95D4F4906877529EC0EA5AF83
2016-05-17 21:29:26,296 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D8B6B648356F412FB3DE9CBDB12E4538, ordinal: 0
2016-05-17 21:29:26,296 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D8B6B648356F412FB3DE9CBDB12E4538, mem on start (mb), free: 300, total: 359, max: 672
2016-05-17 21:29:26,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:26,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:26,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:26,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:26,297 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:26,297 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:26,298 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:26,298 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:26,298 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:26,298 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:26,298 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 68F05DA95D4F4906877529EC0EA5AF83, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:26,298 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A242C9CA4C084292A57A6DCF2C13FC39, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:26,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:26,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 68F05DA95D4F4906877529EC0EA5AF83 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:26,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:26,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:26,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 68F05DA95D4F4906877529EC0EA5AF83
2016-05-17 21:29:26,300 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:26,300 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 68F05DA95D4F4906877529EC0EA5AF83
2016-05-17 21:29:26,300 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:26,301 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520566045_0001_1_00_000000_0
2016-05-17 21:29:26,304 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:26,305 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 68F05DA95D4F4906877529EC0EA5AF83
2016-05-17 21:29:26,305 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 69C94F2D5F3940D98AF1060601EDFA5A
2016-05-17 21:29:26,306 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 69C94F2D5F3940D98AF1060601EDFA5A
2016-05-17 21:29:26,309 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:26,311 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:26,312 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6c7927b7
2016-05-17 21:29:26,313 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6c7927b7
2016-05-17 21:29:26,313 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:26,313 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:26,314 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:26,314 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@61e9704; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@424c8abf; comparator=cascading.tuple.hadoop.util.TupleComparator@2f29239a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:26,315 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:26,315 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:26,316 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:26,329 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:26,329 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:26,329 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:26,329 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:26,330 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:26,330 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:26,331 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:26,331 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D8B6B648356F412FB3DE9CBDB12E4538, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:26,334 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520566045_0001_1_00_000000_0
2016-05-17 21:29:26,334 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D8B6B648356F412FB3DE9CBDB12E4538, mem on close (mb), free: 198, total: 359, max: 672
2016-05-17 21:29:26,334 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:26,334 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:26,335 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:26,335 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:26,341 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:26 Completed running task attempt: attempt_1463520566045_0001_1_00_000000_0
2016-05-17 21:29:26,344 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520566045_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:26,345 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=729, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520566331, Process_Duration=3, Process_End_Time=1463520566334, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:26,345 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520566045_0001_1_00, eventCount=1
2016-05-17 21:29:26,346 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:26,346 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:26,347 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520566045_0001_00_000001
2016-05-17 21:29:26,347 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D8B6B648356F412FB3DE9CBDB12E4538, taskAttemptId=attempt_1463520566045_0001_1_00_000000_0, startTime=1463520566278, finishTime=1463520566346, timeTaken=68, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=729, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520566331, Process_Duration=3, Process_End_Time=1463520566334, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:26,347 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:26,348 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520566045_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:26,348 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:26,348 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520566045_0001_1_00_000000_0
2016-05-17 21:29:26,349 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:26,349 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:TASK_FINISHED]: vertexName=D8B6B648356F412FB3DE9CBDB12E4538, taskId=task_1463520566045_0001_1_00_000000, startTime=1463520566278, finishTime=1463520566348, timeTaken=70, status=SUCCEEDED, successfulAttemptID=attempt_1463520566045_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=729, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520566331, Process_Duration=3, Process_End_Time=1463520566334, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:26,349 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520566045_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:26,350 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538] : 1
2016-05-17 21:29:26,350 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538]
2016-05-17 21:29:26,351 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:26,351 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:VERTEX_FINISHED]: vertexName=D8B6B648356F412FB3DE9CBDB12E4538, vertexId=vertex_1463520566045_0001_1_00, initRequestedTime=1463520566235, initedTime=1463520566252, startRequestedTime=1463520566253, startedTime=1463520566253, finishTime=1463520566350, timeTaken=97, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=729, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520566331, Process_Duration=3, Process_End_Time=1463520566334, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520566278, firstTasksToStart=[ task_1463520566045_0001_1_00_000000 ], lastTaskFinishTime=1463520566346, lastTasksToFinish=[ task_1463520566045_0001_1_00_000000 ], minTaskDuration=68, maxTaskDuration=68, avgTaskDuration=68.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520566045_0001_1_00_000000 ], longestDurationTasks=[ task_1463520566045_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:26,352 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:26,352 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566045_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:26,352 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520566045_0001_00_000001, stoppedTime=1463520566352, exitStatus=0
2016-05-17 21:29:26,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:26,352 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566045_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:26,353 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39] attempt: attempt_1463520566045_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:26,353 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A242C9CA4C084292A57A6DCF2C13FC39
2016-05-17 21:29:26,354 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39]
2016-05-17 21:29:26,354 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:26,354 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520566045_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:26,354 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:TASK_STARTED]: vertexName=A242C9CA4C084292A57A6DCF2C13FC39, taskId=task_1463520566045_0001_1_01_000000, scheduledTime=1463520566354, launchTime=1463520566354
2016-05-17 21:29:26,354 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:26,355 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520566045_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:26,356 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39]
2016-05-17 21:29:26,356 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566045_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:26,356 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A14F8255CD564B9CA3B79372A9B1A6D1/FA73B69024EA4C5AB63E576DDEE9ED98] every/(1/1) ...yplatform/-1_'field'_SWAP, VertexName: A242C9CA4C084292A57A6DCF2C13FC39, VertexParallelism: 1, TaskAttemptID:attempt_1463520566045_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D8B6B648356F412FB3DE9CBDB12E4538, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=AA0AFEB707134C28B90DB99614C34B77, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:26,356 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520566045_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:26,357 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:26,357 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520566045_0001_00_000002
2016-05-17 21:29:26,358 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566045_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:26,358 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520566045_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:26,358 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520566045_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:26,359 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520566045_0001_00_000001, Exiting
2016-05-17 21:29:26,359 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520566045_0001_00_000001
2016-05-17 21:29:26,359 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520566045_0001_00_000001
2016-05-17 21:29:26,359 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520566045_0001_00_000002
2016-05-17 21:29:26,361 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520566045_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:26,361 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:26,361 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520566045_0001_00_000001 completed successfully
2016-05-17 21:29:26,362 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566045_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:26,364 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566045_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:26,364 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:26,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520566045_0001_00_000002, launchTime=1463520566364
2016-05-17 21:29:26,364 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520566045_0001_00_000002
2016-05-17 21:29:26,365 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520566045_0001_1_01_000000_0] to container: [container_1463520566045_0001_00_000002]
2016-05-17 21:29:26,365 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566045_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:26,366 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520566045_0001_00_000002 given task: attempt_1463520566045_0001_1_01_000000_0
2016-05-17 21:29:26,366 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520566045_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520566045_0001_1_01_000000_0
2016-05-17 21:29:26,366 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520566045_0001_1_01_000000_0] started. Is using containerId: [container_1463520566045_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:26,366 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:26,366 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A242C9CA4C084292A57A6DCF2C13FC39, taskAttemptId=attempt_1463520566045_0001_1_01_000000_0, startTime=1463520566366, containerId=container_1463520566045_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520566045_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:26,366 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520566045_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:26,367 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520566045_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:26 Starting to run new task attempt: attempt_1463520566045_0001_1_01_000000_0
2016-05-17 21:29:26,368 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520566045_0001_1_01_000000_0
2016-05-17 21:29:26,369 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:26,369 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:26,370 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:26,370 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:26,370 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:26,371 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:26,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A14F8255CD564B9CA3B79372A9B1A6D1/FA73B69024EA4C5AB63E576DDEE9ED98] every/(1/1) ...yplatform/-1_'field'_SWAP, VertexName: A242C9CA4C084292A57A6DCF2C13FC39, VertexParallelism: 1, TaskAttemptID:attempt_1463520566045_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D8B6B648356F412FB3DE9CBDB12E4538, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=AA0AFEB707134C28B90DB99614C34B77, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:26,372 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:26,373 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520566045_0001_1_01_000000_0
2016-05-17 21:29:26,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:26,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:26,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:26,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AA0AFEB707134C28B90DB99614C34B77, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:26,377 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520566045_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:26,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D8B6B648356F412FB3DE9CBDB12E4538, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:26,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:26,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AA0AFEB707134C28B90DB99614C34B77
2016-05-17 21:29:26,379 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:26,379 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: AA0AFEB707134C28B90DB99614C34B77
2016-05-17 21:29:26,379 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:26,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:26,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D8B6B648356F412FB3DE9CBDB12E4538
2016-05-17 21:29:26,383 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:26,383 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:26,384 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D8B6B648356F412FB3DE9CBDB12E4538
2016-05-17 21:29:26,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D8B6B648356F412FB3DE9CBDB12E4538
2016-05-17 21:29:26,390 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A242C9CA4C084292A57A6DCF2C13FC39, ordinal: 1
2016-05-17 21:29:26,390 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A242C9CA4C084292A57A6DCF2C13FC39, mem on start (mb), free: 191, total: 359, max: 672
2016-05-17 21:29:26,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:26,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:26,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:26,394 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:26,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AA0AFEB707134C28B90DB99614C34B77
2016-05-17 21:29:26,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:26,396 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:26,396 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:26,397 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:26,397 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:26,398 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:26,398 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:26,399 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AA0AFEB707134C28B90DB99614C34B77, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:26,399 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D8B6B648356F412FB3DE9CBDB12E4538, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:26,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:26,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D8B6B648356F412FB3DE9CBDB12E4538 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:26,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:26,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:26,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D8B6B648356F412FB3DE9CBDB12E4538
2016-05-17 21:29:26,401 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:26,402 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:26,402 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D8B6B648356F412FB3DE9CBDB12E4538 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:26,403 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:26,403 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:26,403 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:26,408 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D8B6B648356F412FB3DE9CBDB12E4538
2016-05-17 21:29:26,408 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:26,408 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520566045_0001_1_01_000000_0
2016-05-17 21:29:26,410 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520566045_0001_1_00_000000_0_10026, runDuration: 53678, ]
2016-05-17 21:29:26,411 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:26,411 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520566045_0001_1_00_000000_0_10026] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520566411, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:26,411 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 69C94F2D5F3940D98AF1060601EDFA5A
2016-05-17 21:29:26,412 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D8B6B648356F412FB3DE9CBDB12E4538
2016-05-17 21:29:26,412 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_SWAP"], id: AA0AFEB707134C28B90DB99614C34B77
2016-05-17 21:29:26,412 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:26,412 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_SWAP"]
2016-05-17 21:29:26,412 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D8B6B648356F412FB3DE9CBDB12E4538] #5 in 2ms
2016-05-17 21:29:26,412 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 69C94F2D5F3940D98AF1060601EDFA5A, for 1 inputs
2016-05-17 21:29:26,413 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D8B6B648356F412FB3DE9CBDB12E4538] #5, status:false, isInterrupted:false
2016-05-17 21:29:26,414 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:26,415 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:26,415 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:26,416 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:26,416 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:26,417 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:26,417 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D8B6B648356F412FB3DE9CBDB12E4538
2016-05-17 21:29:26,417 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A242C9CA4C084292A57A6DCF2C13FC39, all 1 inputs ready in: 00:00:00.004
2016-05-17 21:29:26,417 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:26,419 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:26,421 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@740988db; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@1df81d61
2016-05-17 21:29:26,424 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:26,425 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:26,425 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520566045_0001_1_01_000000_0
2016-05-17 21:29:26,426 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520566045_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:26,426 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520566045_0001_1_01_000000_0
2016-05-17 21:29:26,426 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A242C9CA4C084292A57A6DCF2C13FC39, mem on close (mb), free: 186, total: 359, max: 672
2016-05-17 21:29:26,426 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D8B6B648356F412FB3DE9CBDB12E4538
2016-05-17 21:29:26,427 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:26,428 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:26,428 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:26,428 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:26,429 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:26,429 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:26 Completed running task attempt: attempt_1463520566045_0001_1_01_000000_0
2016-05-17 21:29:26,430 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520566045_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:26,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A242C9CA4C084292A57A6DCF2C13FC39, taskAttemptId=attempt_1463520566045_0001_1_01_000000_0, startTime=1463520566366, finishTime=1463520566431, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520566417, Process_Duration=9, Process_End_Time=1463520566426, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:26,432 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520566045_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:26,432 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:26,432 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520566045_0001_1_01_000000_0
2016-05-17 21:29:26,433 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:26,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:TASK_FINISHED]: vertexName=A242C9CA4C084292A57A6DCF2C13FC39, taskId=task_1463520566045_0001_1_01_000000, startTime=1463520566366, finishTime=1463520566432, timeTaken=66, status=SUCCEEDED, successfulAttemptID=attempt_1463520566045_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520566417, Process_Duration=9, Process_End_Time=1463520566426, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:26,434 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520566045_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:26,434 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39] : 1
2016-05-17 21:29:26,434 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39]
2016-05-17 21:29:26,434 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:26,435 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:26,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:VERTEX_FINISHED]: vertexName=A242C9CA4C084292A57A6DCF2C13FC39, vertexId=vertex_1463520566045_0001_1_01, initRequestedTime=1463520566237, initedTime=1463520566247, startRequestedTime=1463520566257, startedTime=1463520566257, finishTime=1463520566434, timeTaken=177, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520566417, Process_Duration=9, Process_End_Time=1463520566426, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520566366, firstTasksToStart=[ task_1463520566045_0001_1_01_000000 ], lastTaskFinishTime=1463520566431, lastTasksToFinish=[ task_1463520566045_0001_1_01_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520566045_0001_1_01_000000 ], longestDurationTasks=[ task_1463520566045_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:26,436 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:26,436 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566045_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:26,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520566417, Process_Duration=9, Process_End_Time=1463520566426, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:26,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:26,436 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:26,437 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:26,436 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520566045_0001_00_000002, stoppedTime=1463520566436, exitStatus=0
2016-05-17 21:29:26,437 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520566045_0001_00_000002
2016-05-17 21:29:26,437 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566045_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:26,438 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520566045_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:26,438 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:26,438 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520566045_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:26,438 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520566045_0001_1
2016-05-17 21:29:26,438 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520566045_0001_00_000002, Exiting
2016-05-17 21:29:26,438 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520566045_0001_00_000002
2016-05-17 21:29:26,439 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520566045_0001_00_000002
2016-05-17 21:29:26,439 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520566045_0001_1
2016-05-17 21:29:26,439 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: AA0AFEB707134C28B90DB99614C34B77 for vertex: vertex_1463520566045_0001_1_01 [A242C9CA4C084292A57A6DCF2C13FC39]
2016-05-17 21:29:26,440 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520566045_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:26,440 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520566045_0001_00_000002 completed successfully
2016-05-17 21:29:26,448 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520566045_0001_1_00 [D8B6B648356F412FB3DE9CBDB12E4538]
2016-05-17 21:29:26,451 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520566045_0001_1, queueSize=0
2016-05-17 21:29:26,451 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:26,452 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566045_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520566045_0001_1, startTime=1463520566235, finishTime=1463520566448, timeTaken=213, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=916, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=753926144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041132748, Process_Duration=12, Process_End_Time=2927041132760, Read_Duration=5, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:26,453 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520566045_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:26,453 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520566045_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:26,454 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520566045_0001_00_000002
2016-05-17 21:29:26,454 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566045_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:26 Completed Dag: dag_1463520566045_0001_1
2016-05-17 21:29:26,457 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520566045_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:26,457 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520566045_0001_1_post
2016-05-17 21:29:26,458 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:26,458 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:26,463 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_'field'_SWAP, applicationId=application_1463520566045_0001
2016-05-17 21:29:26,466 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:26,466 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:26,467 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:26,467 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:26,467 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:26,468 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:26,468 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:26,469 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:26,469 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:26,469 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:26,469 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:26,470 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:26,470 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:26,474 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A14F8255CD564B9CA3B79372A9B1A6D1/FA73B69024EA4C5AB63E576DDEE9ED98/.tez/application_1463520566045_0001
2016-05-17 21:29:26,474 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_SWAP/_temporary
2016-05-17 21:29:26,475 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.452
2016-05-17 21:29:26,544 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_SWAP
2016-05-17 21:29:26,567 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:26,568 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:26,623 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-05-17 21:29:26,624 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-05-17 21:29:26,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:26,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:26,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:26,643 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:26,651 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_-1
2016-05-17 21:29:26,709 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:26,710 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:26,710 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_-1"]
2016-05-17 21:29:26,710 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:26,710 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:26,710 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:26,711 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...blyplatform/-1_UNKNOWN_-1
2016-05-17 21:29:26,711 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:26,724 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:26,729 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:26,730 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:26,730 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:26,739 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0A04725C134348D992B684B84BEAEC37/B402053E2D8342B8B790EFE37516F38A/.tez/application_1463520566729_0001 doesn't exist and is created
2016-05-17 21:29:26,755 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:26,755 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:26,756 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0A04725C134348D992B684B84BEAEC37/B402053E2D8342B8B790EFE37516F38A/.tez/application_1463520566729_0001
2016-05-17 21:29:26,759 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520566729_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:26,770 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:26,770 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:26,773 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:26,773 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:26,773 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0A04725C134348D992B684B84BEAEC37/B402053E2D8342B8B790EFE37516F38A/.tez/application_1463520566729_0001/localmode-log-dir
2016-05-17 21:29:26,773 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0A04725C134348D992B684B84BEAEC37/B402053E2D8342B8B790EFE37516F38A/.tez/application_1463520566729_0001/localmode-log-dir/history.txt.appattempt_1463520566729_0001_000000, maxErrors=10
2016-05-17 21:29:26,774 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:26,774 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520566729_0001_000000, appSubmitTime=1463520566759, launchTime=1463520566759
2016-05-17 21:29:26,777 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:26,779 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:39535
2016-05-17 21:29:26,780 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:26,781 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:26,781 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:26,782 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:26,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520566729_0001_000000, startTime=1463520566784
2016-05-17 21:29:26,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:26,784 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:26,856 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:26,856 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:26,856 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/-1_UNKNOWN_-1, applicationId=application_1463520566729_0001, dagName=[0A04725C134348D992B684B84BEAEC37/B402053E2D8342B8B790EFE37516F38A] every/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-05-17 21:29:26,874 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [0A04725C134348D992B684B84BEAEC37/B402053E2D8342B8B790EFE37516F38A] every/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-05-17 21:29:26,876 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520566729_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0A04725C134348D992B684B84BEAEC37/B402053E2D8342B8B790EFE37516F38A/.tez/application_1463520566729_0001/localmode-log-dir/dag_1463520566729_0001_1.dot
2016-05-17 21:29:26,876 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0A04725C134348D992B684B84BEAEC37/B402053E2D8342B8B790EFE37516F38A/.tez/application_1463520566729_0001/localmode-log-dir/dag_1463520566729_0001_1-tez-dag.pb.txt
2016-05-17 21:29:26,898 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520566729_0001_1
2016-05-17 21:29:26,898 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:26,898 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [0A04725C134348D992B684B84BEAEC37/B402053E2D8342B8B790EFE37516F38A] every/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-05-17 21:29:26 Running Dag: dag_1463520566729_0001_1
2016-05-17 21:29:26,912 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520566729_0001_1, submitTime=1463520566874
2016-05-17 21:29:26,912 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:26,913 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex ABE843C98380420283DC087E764C26CD
2016-05-17 21:29:26,913 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:26,914 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520566729_0001_1, initTime=1463520566912
2016-05-17 21:29:26,914 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:26,914 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520566729_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:26,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:DAG_STARTED]: dagID=dag_1463520566729_0001_1, startTime=1463520566915
2016-05-17 21:29:26,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:26,916 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/-1_UNKNOWN_-1, applicationId=application_1463520566729_0001, dagName=[0A04725C134348D992B684B84BEAEC37/B402053E2D8342B8B790EFE37516F38A] every/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-05-17 21:29:26,916 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:26,917 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520566729_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:26,917 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C0C1F41C0A5D4AF29A8A4CE744E5D3E3 : {ADFCD28E5DA545B183CDB016140807C6={InputName=ADFCD28E5DA545B183CDB016140807C6}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:26,918 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: ADFCD28E5DA545B183CDB016140807C6, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:26,918 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3]
2016-05-17 21:29:26,918 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3]
2016-05-17 21:29:26,918 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3]
2016-05-17 21:29:26,918 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3]. Starting root input initializers: 1
2016-05-17 21:29:26,926 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:26,926 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD]
2016-05-17 21:29:26,929 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:26,929 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD]
2016-05-17 21:29:26,930 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD]
2016-05-17 21:29:26,930 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD]
2016-05-17 21:29:26,930 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=45B413BBD89F43F1AEC2A04B350D063B, vertexId=vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:26,930 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=45B413BBD89F43F1AEC2A04B350D063B, vertexId=vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD]
2016-05-17 21:29:26,934 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: ADFCD28E5DA545B183CDB016140807C6 on vertex vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3]
2016-05-17 21:29:26,936 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=45B413BBD89F43F1AEC2A04B350D063B, vertexName=ABE843C98380420283DC087E764C26CD, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:26,937 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=45B413BBD89F43F1AEC2A04B350D063B, vertexId=vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD]
2016-05-17 21:29:26,939 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520566729_0001, with dag id: dag_1463520566729_0001_1
2016-05-17 21:29:26,940 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:26,940 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ABE843C98380420283DC087E764C26CD, vertexId=vertex_1463520566729_0001_1_01, initRequestedTime=1463520566926, initedTime=1463520566940, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:26,940 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:26,941 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:26,941 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input ADFCD28E5DA545B183CDB016140807C6 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:26,942 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:26,942 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:26,944 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:26,945 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: ADFCD28E5DA545B183CDB016140807C6 on vertex vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3]
2016-05-17 21:29:26,946 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:26,947 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {ADFCD28E5DA545B183CDB016140807C6=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:26,947 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3] parallelism set to 1
2016-05-17 21:29:26,947 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:26,947 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520566729_0001_1_00, eventCount=1
2016-05-17 21:29:26,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C0C1F41C0A5D4AF29A8A4CE744E5D3E3, vertexId=vertex_1463520566729_0001_1_00, initRequestedTime=1463520566917, initedTime=1463520566948, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:26,950 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3] with distanceFromRoot: 0
2016-05-17 21:29:26,950 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:26,950 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C0C1F41C0A5D4AF29A8A4CE744E5D3E3
2016-05-17 21:29:26,951 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3]
2016-05-17 21:29:26,951 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520566729_0001_1_00, startRequestedTime=1463520566950, startedTime=1463520566950
2016-05-17 21:29:26,951 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:26,951 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:TASK_STARTED]: vertexName=C0C1F41C0A5D4AF29A8A4CE744E5D3E3, taskId=task_1463520566729_0001_1_00_000000, scheduledTime=1463520566951, launchTime=1463520566951
2016-05-17 21:29:26,952 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520566729_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:26,952 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520566729_0001_1_00 for vertex: vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD] numStartedSources: 1 numSources: 1
2016-05-17 21:29:26,952 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD] with distanceFromRoot: 1
2016-05-17 21:29:26,952 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3]
2016-05-17 21:29:26,953 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C0C1F41C0A5D4AF29A8A4CE744E5D3E3 in vertex: ABE843C98380420283DC087E764C26CD
2016-05-17 21:29:26,953 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: ABE843C98380420283DC087E764C26CD with 1 source tasks and 1 pending tasks
2016-05-17 21:29:26,953 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520566729_0001_1_01, startRequestedTime=1463520566952, startedTime=1463520566952
2016-05-17 21:29:26,953 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:26,948 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:26,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:26,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:26,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:26,956 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0A04725C134348D992B684B84BEAEC37/B402053E2D8342B8B790EFE37516F38A] every/(1/1) ...blyplatform/-1_UNKNOWN_-1, VertexName: C0C1F41C0A5D4AF29A8A4CE744E5D3E3, VertexParallelism: 1, TaskAttemptID:attempt_1463520566729_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ADFCD28E5DA545B183CDB016140807C6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=ABE843C98380420283DC087E764C26CD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:26,956 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520566729_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:26,956 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:26,965 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:26,965 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:26,966 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:26,966 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520566729_0001_00_000001
2016-05-17 21:29:26,967 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566729_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:26,969 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520566729_0001_00_000001
2016-05-17 21:29:26,969 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:26,970 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566729_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:26,971 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:26,971 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520566729_0001_00_000001, launchTime=1463520566970
2016-05-17 21:29:26,972 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520566729_0001_00_000001
2016-05-17 21:29:26,973 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520566729_0001_1_00_000000_0] to container: [container_1463520566729_0001_00_000001]
2016-05-17 21:29:26,973 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566729_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:26,974 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520566729_0001_1_00_000000_0] started. Is using containerId: [container_1463520566729_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:26,974 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C0C1F41C0A5D4AF29A8A4CE744E5D3E3, taskAttemptId=attempt_1463520566729_0001_1_00_000000_0, startTime=1463520566974, containerId=container_1463520566729_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520566729_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:26,975 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:26,975 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520566729_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:26,975 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520566729_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:26,976 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520566729_0001_00_000001 given task: attempt_1463520566729_0001_1_00_000000_0
2016-05-17 21:29:26,977 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520566729_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520566729_0001_1_00_000000_0
2016-05-17 21:29:26 Starting to run new task attempt: attempt_1463520566729_0001_1_00_000000_0
2016-05-17 21:29:26,977 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520566729_0001_1_00_000000_0
2016-05-17 21:29:26,978 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:26,978 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:26,978 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:26,978 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:26,978 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:26,978 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:26,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0A04725C134348D992B684B84BEAEC37/B402053E2D8342B8B790EFE37516F38A] every/(1/1) ...blyplatform/-1_UNKNOWN_-1, VertexName: C0C1F41C0A5D4AF29A8A4CE744E5D3E3, VertexParallelism: 1, TaskAttemptID:attempt_1463520566729_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ADFCD28E5DA545B183CDB016140807C6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=ABE843C98380420283DC087E764C26CD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:26,979 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:26,980 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520566729_0001_1_00_000000_0
2016-05-17 21:29:26,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:26,981 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:26,984 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=ADFCD28E5DA545B183CDB016140807C6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:26,984 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:26,985 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: ADFCD28E5DA545B183CDB016140807C6
2016-05-17 21:29:26,985 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: ADFCD28E5DA545B183CDB016140807C6
2016-05-17 21:29:26,982 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520566729_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:26,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:26,991 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:26,996 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:26,996 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: ADFCD28E5DA545B183CDB016140807C6
2016-05-17 21:29:26,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: ADFCD28E5DA545B183CDB016140807C6
2016-05-17 21:29:26,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=ABE843C98380420283DC087E764C26CD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:26,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:26,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: ABE843C98380420283DC087E764C26CD
2016-05-17 21:29:27,001 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:27,001 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: ABE843C98380420283DC087E764C26CD
2016-05-17 21:29:27,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: ABE843C98380420283DC087E764C26CD
2016-05-17 21:29:27,007 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C0C1F41C0A5D4AF29A8A4CE744E5D3E3, ordinal: 0
2016-05-17 21:29:27,007 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C0C1F41C0A5D4AF29A8A4CE744E5D3E3, mem on start (mb), free: 289, total: 359, max: 672
2016-05-17 21:29:27,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:27,008 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:27,008 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:27,008 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:27,009 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:27,009 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:27,009 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:27,010 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:27,010 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:27,011 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:27,011 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, ADFCD28E5DA545B183CDB016140807C6, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:27,011 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, ABE843C98380420283DC087E764C26CD, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:27,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:27,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: ADFCD28E5DA545B183CDB016140807C6
2016-05-17 21:29:27,012 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:27,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: ADFCD28E5DA545B183CDB016140807C6
2016-05-17 21:29:27,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: ADFCD28E5DA545B183CDB016140807C6 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:27,014 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:27,014 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:27,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:27,015 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520566729_0001_1_00_000000_0
2016-05-17 21:29:27,018 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:27,018 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:27,019 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: ADFCD28E5DA545B183CDB016140807C6
2016-05-17 21:29:27,020 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 763DA5646BF94EC092284C0D3B12DDE1
2016-05-17 21:29:27,020 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 763DA5646BF94EC092284C0D3B12DDE1
2016-05-17 21:29:27,020 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6a08648
2016-05-17 21:29:27,022 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:27,023 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6a08648
2016-05-17 21:29:27,024 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:27,025 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:27,027 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:27,027 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@14cfe45e; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@5c99e1fe; comparator=cascading.tuple.hadoop.util.TupleComparator@74d182d9; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:27,028 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:27,029 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:27,030 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:27,043 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:27,043 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:27,044 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:27,044 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:27,045 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:27,046 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:27,046 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:27,046 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C0C1F41C0A5D4AF29A8A4CE744E5D3E3, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:27,050 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520566729_0001_1_00_000000_0
2016-05-17 21:29:27,050 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C0C1F41C0A5D4AF29A8A4CE744E5D3E3, mem on close (mb), free: 187, total: 359, max: 672
2016-05-17 21:29:27,050 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:27,051 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:27,051 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:27,051 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:27,056 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:27 Completed running task attempt: attempt_1463520566729_0001_1_00_000000_0
2016-05-17 21:29:27,061 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520566729_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:27,063 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520566729_0001_1_00, eventCount=1
2016-05-17 21:29:27,063 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:27,063 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C0C1F41C0A5D4AF29A8A4CE744E5D3E3, taskAttemptId=attempt_1463520566729_0001_1_00_000000_0, startTime=1463520566974, finishTime=1463520567063, timeTaken=89, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376438784, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520567046, Process_Duration=4, Process_End_Time=1463520567050, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:27,064 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520566729_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:27,064 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:27,064 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520566729_0001_1_00_000000_0
2016-05-17 21:29:27,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:TASK_FINISHED]: vertexName=C0C1F41C0A5D4AF29A8A4CE744E5D3E3, taskId=task_1463520566729_0001_1_00_000000, startTime=1463520566974, finishTime=1463520567064, timeTaken=90, status=SUCCEEDED, successfulAttemptID=attempt_1463520566729_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376438784, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520567046, Process_Duration=4, Process_End_Time=1463520567050, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:27,065 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520566729_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:27,065 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3] : 1
2016-05-17 21:29:27,065 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3]
2016-05-17 21:29:27,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:27,067 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:VERTEX_FINISHED]: vertexName=C0C1F41C0A5D4AF29A8A4CE744E5D3E3, vertexId=vertex_1463520566729_0001_1_00, initRequestedTime=1463520566917, initedTime=1463520566948, startRequestedTime=1463520566950, startedTime=1463520566950, finishTime=1463520567065, timeTaken=115, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376438784, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520567046, Process_Duration=4, Process_End_Time=1463520567050, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520566974, firstTasksToStart=[ task_1463520566729_0001_1_00_000000 ], lastTaskFinishTime=1463520567063, lastTasksToFinish=[ task_1463520566729_0001_1_00_000000 ], minTaskDuration=89, maxTaskDuration=89, avgTaskDuration=89.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520566729_0001_1_00_000000 ], longestDurationTasks=[ task_1463520566729_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:27,068 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:27,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376438784, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520567046, Process_Duration=4, Process_End_Time=1463520567050, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:27,068 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566729_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:27,068 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD] attempt: attempt_1463520566729_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:27,069 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: ABE843C98380420283DC087E764C26CD
2016-05-17 21:29:27,069 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD]
2016-05-17 21:29:27,069 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520566729_0001_00_000001, stoppedTime=1463520567069, exitStatus=0
2016-05-17 21:29:27,069 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566729_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:27,069 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:27,070 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:TASK_STARTED]: vertexName=ABE843C98380420283DC087E764C26CD, taskId=task_1463520566729_0001_1_01_000000, scheduledTime=1463520567070, launchTime=1463520567070
2016-05-17 21:29:27,070 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520566729_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:27,070 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520566729_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:27,071 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD]
2016-05-17 21:29:27,071 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566729_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:27,071 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0A04725C134348D992B684B84BEAEC37/B402053E2D8342B8B790EFE37516F38A] every/(1/1) ...blyplatform/-1_UNKNOWN_-1, VertexName: ABE843C98380420283DC087E764C26CD, VertexParallelism: 1, TaskAttemptID:attempt_1463520566729_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C0C1F41C0A5D4AF29A8A4CE744E5D3E3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=45B413BBD89F43F1AEC2A04B350D063B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:27,072 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520566729_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:27,068 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:27,072 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:27,073 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:27,073 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:27,075 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520566729_0001_00_000002
2016-05-17 21:29:27,075 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:27,076 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:27,076 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520566729_0001_00_000001
2016-05-17 21:29:27,076 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520566729_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:27,077 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520566729_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:27,077 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566729_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:27,077 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520566729_0001_00_000001, Exiting
2016-05-17 21:29:27,078 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520566729_0001_00_000001
2016-05-17 21:29:27,078 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520566729_0001_00_000001
2016-05-17 21:29:27,087 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520566729_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:27,088 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520566729_0001_00_000001 completed successfully
2016-05-17 21:29:27,089 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520566729_0001_00_000002
2016-05-17 21:29:27,090 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:27,091 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566729_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:27,093 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566729_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:27,094 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:27,094 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520566729_0001_00_000002, launchTime=1463520567094
2016-05-17 21:29:27,095 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520566729_0001_00_000002
2016-05-17 21:29:27,095 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520566729_0001_1_01_000000_0] to container: [container_1463520566729_0001_00_000002]
2016-05-17 21:29:27,095 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566729_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:27,096 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520566729_0001_1_01_000000_0] started. Is using containerId: [container_1463520566729_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:27,096 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ABE843C98380420283DC087E764C26CD, taskAttemptId=attempt_1463520566729_0001_1_01_000000_0, startTime=1463520567096, containerId=container_1463520566729_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520566729_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:27,096 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520566729_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:27,097 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520566729_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:27,097 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:27,097 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520566729_0001_00_000002 given task: attempt_1463520566729_0001_1_01_000000_0
2016-05-17 21:29:27,097 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520566729_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520566729_0001_1_01_000000_0
2016-05-17 21:29:27 Starting to run new task attempt: attempt_1463520566729_0001_1_01_000000_0
2016-05-17 21:29:27,098 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520566729_0001_1_01_000000_0
2016-05-17 21:29:27,098 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:27,098 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:27,098 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:27,098 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:27,099 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:27,099 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:27,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0A04725C134348D992B684B84BEAEC37/B402053E2D8342B8B790EFE37516F38A] every/(1/1) ...blyplatform/-1_UNKNOWN_-1, VertexName: ABE843C98380420283DC087E764C26CD, VertexParallelism: 1, TaskAttemptID:attempt_1463520566729_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C0C1F41C0A5D4AF29A8A4CE744E5D3E3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=45B413BBD89F43F1AEC2A04B350D063B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:27,099 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:27,104 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520566729_0001_1_01_000000_0
2016-05-17 21:29:27,104 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:27,104 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:27,105 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C0C1F41C0A5D4AF29A8A4CE744E5D3E3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:27,106 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:27,106 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C0C1F41C0A5D4AF29A8A4CE744E5D3E3
2016-05-17 21:29:27,108 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:27,108 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:27,109 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C0C1F41C0A5D4AF29A8A4CE744E5D3E3
2016-05-17 21:29:27,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C0C1F41C0A5D4AF29A8A4CE744E5D3E3
2016-05-17 21:29:27,109 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:27,112 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520566729_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:27,114 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:27,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=45B413BBD89F43F1AEC2A04B350D063B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:27,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:27,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 45B413BBD89F43F1AEC2A04B350D063B
2016-05-17 21:29:27,116 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:27,116 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 45B413BBD89F43F1AEC2A04B350D063B
2016-05-17 21:29:27,124 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ABE843C98380420283DC087E764C26CD, ordinal: 1
2016-05-17 21:29:27,124 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ABE843C98380420283DC087E764C26CD, mem on start (mb), free: 179, total: 359, max: 672
2016-05-17 21:29:27,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:27,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:27,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:27,129 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:27,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 45B413BBD89F43F1AEC2A04B350D063B
2016-05-17 21:29:27,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:27,130 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:27,130 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:27,130 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:27,130 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:27,130 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:27,131 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:27,131 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C0C1F41C0A5D4AF29A8A4CE744E5D3E3, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:27,131 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 45B413BBD89F43F1AEC2A04B350D063B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:27,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:27,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C0C1F41C0A5D4AF29A8A4CE744E5D3E3 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:27,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:27,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:27,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C0C1F41C0A5D4AF29A8A4CE744E5D3E3
2016-05-17 21:29:27,133 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:27,133 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:27,133 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C0C1F41C0A5D4AF29A8A4CE744E5D3E3 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:27,134 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:27,134 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:27,134 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:27,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C0C1F41C0A5D4AF29A8A4CE744E5D3E3
2016-05-17 21:29:27,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:27,136 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520566729_0001_1_01_000000_0
2016-05-17 21:29:27,139 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:27,140 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 763DA5646BF94EC092284C0D3B12DDE1
2016-05-17 21:29:27,141 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_-1"], id: 45B413BBD89F43F1AEC2A04B350D063B
2016-05-17 21:29:27,141 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_-1"]
2016-05-17 21:29:27,141 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 763DA5646BF94EC092284C0D3B12DDE1, for 1 inputs
2016-05-17 21:29:27,142 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520566729_0001_1_00_000000_0_10033, runDuration: 62005, ]
2016-05-17 21:29:27,144 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520566729_0001_1_00_000000_0_10033] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520567143, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:27,144 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C0C1F41C0A5D4AF29A8A4CE744E5D3E3
2016-05-17 21:29:27,144 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:27,144 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [C0C1F41C0A5D4AF29A8A4CE744E5D3E3] #6 in 2ms
2016-05-17 21:29:27,144 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C0C1F41C0A5D4AF29A8A4CE744E5D3E3] #6, status:false, isInterrupted:false
2016-05-17 21:29:27,145 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:27,147 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:27,147 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:27,147 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:27,147 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:27,148 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:27,148 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C0C1F41C0A5D4AF29A8A4CE744E5D3E3
2016-05-17 21:29:27,148 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ABE843C98380420283DC087E764C26CD, all 1 inputs ready in: 00:00:00.007
2016-05-17 21:29:27,150 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:27,151 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:27,154 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@35a1b58a; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@b959288
2016-05-17 21:29:27,158 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:27,158 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:27,159 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520566729_0001_1_01_000000_0
2016-05-17 21:29:27,159 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520566729_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:27,160 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520566729_0001_1_01_000000_0
2016-05-17 21:29:27,160 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ABE843C98380420283DC087E764C26CD, mem on close (mb), free: 175, total: 359, max: 672
2016-05-17 21:29:27,161 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C0C1F41C0A5D4AF29A8A4CE744E5D3E3
2016-05-17 21:29:27,161 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:27,163 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:27,163 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:27,163 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:27,163 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:27,164 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:27 Completed running task attempt: attempt_1463520566729_0001_1_01_000000_0
2016-05-17 21:29:27,166 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520566729_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:27,167 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376438784, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520567149, Process_Duration=11, Process_End_Time=1463520567160, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:27,168 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ABE843C98380420283DC087E764C26CD, taskAttemptId=attempt_1463520566729_0001_1_01_000000_0, startTime=1463520567096, finishTime=1463520567167, timeTaken=71, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376438784, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520567149, Process_Duration=11, Process_End_Time=1463520567160, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:27,168 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520566729_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:27,168 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:27,167 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:27,169 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520566729_0001_1_01_000000_0
2016-05-17 21:29:27,170 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:27,170 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:TASK_FINISHED]: vertexName=ABE843C98380420283DC087E764C26CD, taskId=task_1463520566729_0001_1_01_000000, startTime=1463520567096, finishTime=1463520567169, timeTaken=73, status=SUCCEEDED, successfulAttemptID=attempt_1463520566729_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376438784, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520567149, Process_Duration=11, Process_End_Time=1463520567160, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:27,170 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520566729_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:27,171 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD] : 1
2016-05-17 21:29:27,171 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD]
2016-05-17 21:29:27,172 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:27,168 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:27,173 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:27,172 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:VERTEX_FINISHED]: vertexName=ABE843C98380420283DC087E764C26CD, vertexId=vertex_1463520566729_0001_1_01, initRequestedTime=1463520566926, initedTime=1463520566940, startRequestedTime=1463520566952, startedTime=1463520566952, finishTime=1463520567171, timeTaken=219, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376438784, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520567149, Process_Duration=11, Process_End_Time=1463520567160, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520567096, firstTasksToStart=[ task_1463520566729_0001_1_01_000000 ], lastTaskFinishTime=1463520567167, lastTasksToFinish=[ task_1463520566729_0001_1_01_000000 ], minTaskDuration=71, maxTaskDuration=71, avgTaskDuration=71.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520566729_0001_1_01_000000 ], longestDurationTasks=[ task_1463520566729_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:27,173 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:27,173 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566729_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:27,173 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520566729_0001_00_000002, stoppedTime=1463520567173, exitStatus=0
2016-05-17 21:29:27,174 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566729_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:27,174 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:27,174 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520566729_0001_1
2016-05-17 21:29:27,174 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520566729_0001_1
2016-05-17 21:29:27,175 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520566729_0001_1_00 [C0C1F41C0A5D4AF29A8A4CE744E5D3E3]
2016-05-17 21:29:27,175 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 45B413BBD89F43F1AEC2A04B350D063B for vertex: vertex_1463520566729_0001_1_01 [ABE843C98380420283DC087E764C26CD]
2016-05-17 21:29:27,175 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:27,176 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520566729_0001_00_000002
2016-05-17 21:29:27,176 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520566729_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:27,177 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520566729_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:27,177 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520566729_0001_00_000002, Exiting
2016-05-17 21:29:27,177 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520566729_0001_00_000002
2016-05-17 21:29:27,177 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520566729_0001_00_000002
2016-05-17 21:29:27,183 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520566729_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:27,183 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520566729_0001_00_000002 completed successfully
2016-05-17 21:29:27,194 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520566729_0001_1, queueSize=0
2016-05-17 21:29:27,195 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:27,196 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520566729_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520566729_0001_1, startTime=1463520566915, finishTime=1463520567191, timeTaken=276, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=345, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=752877568, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041134195, Process_Duration=15, Process_End_Time=2927041134210, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:27,196 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520566729_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:27,196 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520566729_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:27,196 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520566729_0001_00_000002
2016-05-17 21:29:27,196 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520566729_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:27 Completed Dag: dag_1463520566729_0001_1
2016-05-17 21:29:27,197 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520566729_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:27,197 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520566729_0001_1_post
2016-05-17 21:29:27,197 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:27,197 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:27,207 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/-1_UNKNOWN_-1, applicationId=application_1463520566729_0001
2016-05-17 21:29:27,210 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:27,210 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:27,210 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:27,211 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:27,211 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:27,212 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:27,212 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:27,213 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:27,214 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:27,214 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:27,214 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:27,214 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:27,215 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:27,221 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0A04725C134348D992B684B84BEAEC37/B402053E2D8342B8B790EFE37516F38A/.tez/application_1463520566729_0001
2016-05-17 21:29:27,221 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_-1/_temporary
2016-05-17 21:29:27,222 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.512
2016-05-17 21:29:27,327 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_-1
2016-05-17 21:29:27,350 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:27,351 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:27,383 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-17 21:29:27,387 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.010
2016-05-17 21:29:27,387 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:27,387 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:27,387 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_'field'
2016-05-17 21:29:27,421 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:27,421 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:27,486 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-17 21:29:27,499 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.033
2016-05-17 21:29:27,499 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:27,499 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:27,500 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:27,516 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:27,525 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_RESULTS
2016-05-17 21:29:27,576 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:27,576 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:27,577 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_RESULTS"]
2016-05-17 21:29:27,577 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:27,577 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:27,577 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:27,578 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...atform/-1_UNKNOWN_RESULTS
2016-05-17 21:29:27,578 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:27,597 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:27,603 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:27,604 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:27,604 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:27,617 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52712520A8C3455D9480755A1075DBC3/CA084EFEB4CD48379DE655DBDADA7216/.tez/application_1463520567603_0001 doesn't exist and is created
2016-05-17 21:29:27,639 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:27,639 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:27,641 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52712520A8C3455D9480755A1075DBC3/CA084EFEB4CD48379DE655DBDADA7216/.tez/application_1463520567603_0001
2016-05-17 21:29:27,647 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520567603_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:27,664 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:27,666 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:27,668 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:27,669 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:27,669 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52712520A8C3455D9480755A1075DBC3/CA084EFEB4CD48379DE655DBDADA7216/.tez/application_1463520567603_0001/localmode-log-dir
2016-05-17 21:29:27,670 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52712520A8C3455D9480755A1075DBC3/CA084EFEB4CD48379DE655DBDADA7216/.tez/application_1463520567603_0001/localmode-log-dir/history.txt.appattempt_1463520567603_0001_000000, maxErrors=10
2016-05-17 21:29:27,671 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:27,671 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520567603_0001_000000, appSubmitTime=1463520567646, launchTime=1463520567646
2016-05-17 21:29:27,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:27,679 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:27,682 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:27,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:27,687 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56455
2016-05-17 21:29:27,688 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:27,692 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520567603_0001_000000, startTime=1463520567692
2016-05-17 21:29:27,692 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:27,692 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:27,740 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:27,740 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:27,740 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_RESULTS, applicationId=application_1463520567603_0001, dagName=[52712520A8C3455D9480755A1075DBC3/CA084EFEB4CD48379DE655DBDADA7216] every/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-05-17 21:29:27,758 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [52712520A8C3455D9480755A1075DBC3/CA084EFEB4CD48379DE655DBDADA7216] every/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-05-17 21:29:27,759 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520567603_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52712520A8C3455D9480755A1075DBC3/CA084EFEB4CD48379DE655DBDADA7216/.tez/application_1463520567603_0001/localmode-log-dir/dag_1463520567603_0001_1.dot
2016-05-17 21:29:27,760 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52712520A8C3455D9480755A1075DBC3/CA084EFEB4CD48379DE655DBDADA7216/.tez/application_1463520567603_0001/localmode-log-dir/dag_1463520567603_0001_1-tez-dag.pb.txt
2016-05-17 21:29:27,770 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520567603_0001_1
2016-05-17 21:29:27,770 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:27,770 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [52712520A8C3455D9480755A1075DBC3/CA084EFEB4CD48379DE655DBDADA7216] every/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-05-17 21:29:27 Running Dag: dag_1463520567603_0001_1
2016-05-17 21:29:27,782 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520567603_0001_1, submitTime=1463520567758
2016-05-17 21:29:27,782 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:27,783 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DBB52415DED3404EB2A1C730C15BFF76
2016-05-17 21:29:27,783 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:27,783 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:27,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520567603_0001_1, initTime=1463520567782
2016-05-17 21:29:27,785 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520567603_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:27,785 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:DAG_STARTED]: dagID=dag_1463520567603_0001_1, startTime=1463520567785
2016-05-17 21:29:27,785 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:27,786 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:27,786 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520567603_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:27,786 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A6197E416AF04ED8A839C3D630E76C56 : {ABF051900B5F49B1B5AAE165C1F7217D={InputName=ABF051900B5F49B1B5AAE165C1F7217D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:27,786 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: ABF051900B5F49B1B5AAE165C1F7217D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:27,786 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56]
2016-05-17 21:29:27,787 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56]
2016-05-17 21:29:27,787 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56]
2016-05-17 21:29:27,787 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56]. Starting root input initializers: 1
2016-05-17 21:29:27,788 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:27,788 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_RESULTS, applicationId=application_1463520567603_0001, dagName=[52712520A8C3455D9480755A1075DBC3/CA084EFEB4CD48379DE655DBDADA7216] every/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-05-17 21:29:27,788 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76]
2016-05-17 21:29:27,789 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: ABF051900B5F49B1B5AAE165C1F7217D on vertex vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56]
2016-05-17 21:29:27,790 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:27,791 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input ABF051900B5F49B1B5AAE165C1F7217D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:27,791 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:27,791 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76]
2016-05-17 21:29:27,791 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76]
2016-05-17 21:29:27,791 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76]
2016-05-17 21:29:27,791 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9CC4C6D8F38B49198774F4D3D9652D95, vertexId=vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:27,791 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9CC4C6D8F38B49198774F4D3D9652D95, vertexId=vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76]
2016-05-17 21:29:27,791 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:27,793 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9CC4C6D8F38B49198774F4D3D9652D95, vertexName=DBB52415DED3404EB2A1C730C15BFF76, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:27,794 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9CC4C6D8F38B49198774F4D3D9652D95, vertexId=vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76]
2016-05-17 21:29:27,795 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:27,793 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:27,795 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DBB52415DED3404EB2A1C730C15BFF76, vertexId=vertex_1463520567603_0001_1_01, initRequestedTime=1463520567788, initedTime=1463520567795, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:27,797 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:27,798 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:27,798 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: ABF051900B5F49B1B5AAE165C1F7217D on vertex vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56]
2016-05-17 21:29:27,798 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:27,799 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {ABF051900B5F49B1B5AAE165C1F7217D=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:27,799 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56] parallelism set to 1
2016-05-17 21:29:27,800 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:27,801 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520567603_0001_1_00, eventCount=1
2016-05-17 21:29:27,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:27,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A6197E416AF04ED8A839C3D630E76C56, vertexId=vertex_1463520567603_0001_1_00, initRequestedTime=1463520567786, initedTime=1463520567801, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:27,802 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56] with distanceFromRoot: 0
2016-05-17 21:29:27,803 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:27,803 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A6197E416AF04ED8A839C3D630E76C56
2016-05-17 21:29:27,804 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56]
2016-05-17 21:29:27,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:27,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520567603_0001_1_00, startRequestedTime=1463520567803, startedTime=1463520567803
2016-05-17 21:29:27,805 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:27,806 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:27,806 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:TASK_STARTED]: vertexName=A6197E416AF04ED8A839C3D630E76C56, taskId=task_1463520567603_0001_1_00_000000, scheduledTime=1463520567806, launchTime=1463520567806
2016-05-17 21:29:27,807 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520567603_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:27,807 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520567603_0001_1_00 for vertex: vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76] numStartedSources: 1 numSources: 1
2016-05-17 21:29:27,808 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76] with distanceFromRoot: 1
2016-05-17 21:29:27,808 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56]
2016-05-17 21:29:27,809 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A6197E416AF04ED8A839C3D630E76C56 in vertex: DBB52415DED3404EB2A1C730C15BFF76
2016-05-17 21:29:27,809 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DBB52415DED3404EB2A1C730C15BFF76 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:27,809 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:27,810 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520567603_0001_1_01, startRequestedTime=1463520567808, startedTime=1463520567808
2016-05-17 21:29:27,810 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:27,811 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [52712520A8C3455D9480755A1075DBC3/CA084EFEB4CD48379DE655DBDADA7216] every/(1/1) ...atform/-1_UNKNOWN_RESULTS, VertexName: A6197E416AF04ED8A839C3D630E76C56, VertexParallelism: 1, TaskAttemptID:attempt_1463520567603_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ABF051900B5F49B1B5AAE165C1F7217D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DBB52415DED3404EB2A1C730C15BFF76, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:27,811 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520567603_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:27,812 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:27,813 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:27,813 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:27,813 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:27,814 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520567603_0001_00_000001
2016-05-17 21:29:27,815 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520567603_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:27,817 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520567603_0001_00_000001
2016-05-17 21:29:27,817 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:27,818 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520567603_0001, with dag id: dag_1463520567603_0001_1
2016-05-17 21:29:27,822 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520567603_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:27,822 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520567603_0001_00_000001, launchTime=1463520567822
2016-05-17 21:29:27,822 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:27,823 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520567603_0001_00_000001
2016-05-17 21:29:27,824 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520567603_0001_1_00_000000_0] to container: [container_1463520567603_0001_00_000001]
2016-05-17 21:29:27,825 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520567603_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:27,825 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520567603_0001_00_000001 given task: attempt_1463520567603_0001_1_00_000000_0
2016-05-17 21:29:27,825 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520567603_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520567603_0001_1_00_000000_0
2016-05-17 21:29:27,825 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520567603_0001_1_00_000000_0] started. Is using containerId: [container_1463520567603_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:27,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A6197E416AF04ED8A839C3D630E76C56, taskAttemptId=attempt_1463520567603_0001_1_00_000000_0, startTime=1463520567825, containerId=container_1463520567603_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520567603_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:27,826 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520567603_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:27,826 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520567603_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:27,826 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:27 Starting to run new task attempt: attempt_1463520567603_0001_1_00_000000_0
2016-05-17 21:29:27,827 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520567603_0001_1_00_000000_0
2016-05-17 21:29:27,827 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:27,827 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:27,828 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:27,828 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:27,828 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:27,828 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:27,828 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [52712520A8C3455D9480755A1075DBC3/CA084EFEB4CD48379DE655DBDADA7216] every/(1/1) ...atform/-1_UNKNOWN_RESULTS, VertexName: A6197E416AF04ED8A839C3D630E76C56, VertexParallelism: 1, TaskAttemptID:attempt_1463520567603_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ABF051900B5F49B1B5AAE165C1F7217D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DBB52415DED3404EB2A1C730C15BFF76, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:27,828 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:27,829 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520567603_0001_1_00_000000_0
2016-05-17 21:29:27,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:27,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:27,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=ABF051900B5F49B1B5AAE165C1F7217D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:27,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:27,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: ABF051900B5F49B1B5AAE165C1F7217D
2016-05-17 21:29:27,831 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: ABF051900B5F49B1B5AAE165C1F7217D
2016-05-17 21:29:27,833 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:27,833 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:27,833 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520567603_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:27,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DBB52415DED3404EB2A1C730C15BFF76, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:27,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:27,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DBB52415DED3404EB2A1C730C15BFF76
2016-05-17 21:29:27,833 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: ABF051900B5F49B1B5AAE165C1F7217D
2016-05-17 21:29:27,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: ABF051900B5F49B1B5AAE165C1F7217D
2016-05-17 21:29:27,836 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:27,836 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DBB52415DED3404EB2A1C730C15BFF76
2016-05-17 21:29:27,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DBB52415DED3404EB2A1C730C15BFF76
2016-05-17 21:29:27,837 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:27,843 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A6197E416AF04ED8A839C3D630E76C56, ordinal: 0
2016-05-17 21:29:27,843 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A6197E416AF04ED8A839C3D630E76C56, mem on start (mb), free: 270, total: 359, max: 672
2016-05-17 21:29:27,844 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:27,844 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:27,844 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:27,844 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:27,845 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:27,845 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:27,845 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:27,845 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:27,846 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:27,846 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:27,846 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, ABF051900B5F49B1B5AAE165C1F7217D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:27,846 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DBB52415DED3404EB2A1C730C15BFF76, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:27,846 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:27,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: ABF051900B5F49B1B5AAE165C1F7217D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:27,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:27,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:27,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: ABF051900B5F49B1B5AAE165C1F7217D
2016-05-17 21:29:27,847 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:27,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: ABF051900B5F49B1B5AAE165C1F7217D
2016-05-17 21:29:27,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:27,848 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520567603_0001_1_00_000000_0
2016-05-17 21:29:27,848 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:27,849 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3c2153f6
2016-05-17 21:29:27,850 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3c2153f6
2016-05-17 21:29:27,850 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:27,850 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:27,851 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:27,852 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: ABF051900B5F49B1B5AAE165C1F7217D
2016-05-17 21:29:27,852 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: EB8736F2AB694002BD1B995DBF595BF0
2016-05-17 21:29:27,853 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] EB8736F2AB694002BD1B995DBF595BF0
2016-05-17 21:29:27,854 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:27,855 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:27,855 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:27,858 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:27,858 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@61d7499d; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@7d221499; comparator=cascading.tuple.hadoop.util.TupleComparator@5e1bbf35; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:27,858 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:27,858 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:27,858 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:27,871 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:27,871 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:27,872 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:27,872 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:27,872 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:27,873 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:27,873 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:27,874 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A6197E416AF04ED8A839C3D630E76C56, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:27,876 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520567603_0001_1_00_000000_0
2016-05-17 21:29:27,877 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A6197E416AF04ED8A839C3D630E76C56, mem on close (mb), free: 167, total: 359, max: 672
2016-05-17 21:29:27,877 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:27,877 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:27,877 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:27,878 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:27,882 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:27 Completed running task attempt: attempt_1463520567603_0001_1_00_000000_0
2016-05-17 21:29:27,886 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520567603_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:27,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520567603_0001_1_00, eventCount=1
2016-05-17 21:29:27,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A6197E416AF04ED8A839C3D630E76C56, taskAttemptId=attempt_1463520567603_0001_1_00_000000_0, startTime=1463520567825, finishTime=1463520567887, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520567874, Process_Duration=2, Process_End_Time=1463520567876, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:27,888 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520567603_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:27,888 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:27,888 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520567603_0001_1_00_000000_0
2016-05-17 21:29:27,889 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:27,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:TASK_FINISHED]: vertexName=A6197E416AF04ED8A839C3D630E76C56, taskId=task_1463520567603_0001_1_00_000000, startTime=1463520567825, finishTime=1463520567888, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1463520567603_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520567874, Process_Duration=2, Process_End_Time=1463520567876, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:27,889 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520567603_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:27,889 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56] : 1
2016-05-17 21:29:27,889 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56]
2016-05-17 21:29:27,888 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:27,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520567874, Process_Duration=2, Process_End_Time=1463520567876, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:27,892 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:27,892 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:VERTEX_FINISHED]: vertexName=A6197E416AF04ED8A839C3D630E76C56, vertexId=vertex_1463520567603_0001_1_00, initRequestedTime=1463520567786, initedTime=1463520567801, startRequestedTime=1463520567803, startedTime=1463520567803, finishTime=1463520567889, timeTaken=86, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520567874, Process_Duration=2, Process_End_Time=1463520567876, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520567825, firstTasksToStart=[ task_1463520567603_0001_1_00_000000 ], lastTaskFinishTime=1463520567887, lastTasksToFinish=[ task_1463520567603_0001_1_00_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520567603_0001_1_00_000000 ], longestDurationTasks=[ task_1463520567603_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:27,892 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:27,893 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:27,893 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520567603_0001_00_000001
2016-05-17 21:29:27,893 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:27,894 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76] attempt: attempt_1463520567603_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:27,895 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DBB52415DED3404EB2A1C730C15BFF76
2016-05-17 21:29:27,895 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76]
2016-05-17 21:29:27,895 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520567603_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:27,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:27,895 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520567603_0001_00_000001, stoppedTime=1463520567895, exitStatus=0
2016-05-17 21:29:27,896 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520567603_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:27,896 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:27,897 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:27,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:TASK_STARTED]: vertexName=DBB52415DED3404EB2A1C730C15BFF76, taskId=task_1463520567603_0001_1_01_000000, scheduledTime=1463520567897, launchTime=1463520567897
2016-05-17 21:29:27,898 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520567603_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:27,898 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520567603_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:27,898 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76]
2016-05-17 21:29:27,898 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520567603_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:27,899 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [52712520A8C3455D9480755A1075DBC3/CA084EFEB4CD48379DE655DBDADA7216] every/(1/1) ...atform/-1_UNKNOWN_RESULTS, VertexName: DBB52415DED3404EB2A1C730C15BFF76, VertexParallelism: 1, TaskAttemptID:attempt_1463520567603_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A6197E416AF04ED8A839C3D630E76C56, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9CC4C6D8F38B49198774F4D3D9652D95, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:27,899 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520567603_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:27,899 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:27,900 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520567603_0001_00_000002
2016-05-17 21:29:27,900 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520567603_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:27,901 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520567603_0001_00_000002
2016-05-17 21:29:27,901 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:27,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:27,902 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520567603_0001_00_000002, launchTime=1463520567902
2016-05-17 21:29:27,902 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520567603_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:27,903 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520567603_0001_00_000002
2016-05-17 21:29:27,903 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520567603_0001_1_01_000000_0] to container: [container_1463520567603_0001_00_000002]
2016-05-17 21:29:27,904 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520567603_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:27,904 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520567603_0001_00_000002 given task: attempt_1463520567603_0001_1_01_000000_0
2016-05-17 21:29:27,904 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520567603_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520567603_0001_1_01_000000_0
2016-05-17 21:29:27,904 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520567603_0001_1_01_000000_0] started. Is using containerId: [container_1463520567603_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:27,905 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:27,905 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520567603_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:27,905 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520567603_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:27,905 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DBB52415DED3404EB2A1C730C15BFF76, taskAttemptId=attempt_1463520567603_0001_1_01_000000_0, startTime=1463520567904, containerId=container_1463520567603_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520567603_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:27 Starting to run new task attempt: attempt_1463520567603_0001_1_01_000000_0
2016-05-17 21:29:27,905 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520567603_0001_00_000001, Exiting
2016-05-17 21:29:27,905 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520567603_0001_1_01_000000_0
2016-05-17 21:29:27,905 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520567603_0001_00_000001
2016-05-17 21:29:27,905 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520567603_0001_00_000001
2016-05-17 21:29:27,905 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:27,907 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:27,908 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:27,908 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:27,907 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520567603_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:27,908 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:27,908 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520567603_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:27,909 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520567603_0001_00_000001 completed successfully
2016-05-17 21:29:27,909 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:27,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [52712520A8C3455D9480755A1075DBC3/CA084EFEB4CD48379DE655DBDADA7216] every/(1/1) ...atform/-1_UNKNOWN_RESULTS, VertexName: DBB52415DED3404EB2A1C730C15BFF76, VertexParallelism: 1, TaskAttemptID:attempt_1463520567603_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A6197E416AF04ED8A839C3D630E76C56, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9CC4C6D8F38B49198774F4D3D9652D95, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:27,910 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:27,909 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520567603_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:27,911 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520567603_0001_1_01_000000_0
2016-05-17 21:29:27,914 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520567603_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:27,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:27,915 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:27,915 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520567603_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:27,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:27,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A6197E416AF04ED8A839C3D630E76C56, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:27,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:27,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A6197E416AF04ED8A839C3D630E76C56
2016-05-17 21:29:27,918 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:27,918 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:27,918 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A6197E416AF04ED8A839C3D630E76C56
2016-05-17 21:29:27,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A6197E416AF04ED8A839C3D630E76C56
2016-05-17 21:29:27,919 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:27,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9CC4C6D8F38B49198774F4D3D9652D95, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:27,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:27,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9CC4C6D8F38B49198774F4D3D9652D95
2016-05-17 21:29:27,920 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:27,920 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9CC4C6D8F38B49198774F4D3D9652D95
2016-05-17 21:29:27,932 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:27,932 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DBB52415DED3404EB2A1C730C15BFF76, ordinal: 1
2016-05-17 21:29:27,932 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DBB52415DED3404EB2A1C730C15BFF76, mem on start (mb), free: 160, total: 359, max: 672
2016-05-17 21:29:27,932 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:27,932 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:27,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:27,932 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9CC4C6D8F38B49198774F4D3D9652D95
2016-05-17 21:29:27,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:27,933 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:27,935 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:27,935 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:27,935 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:27,935 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:27,936 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:27,936 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A6197E416AF04ED8A839C3D630E76C56, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:27,936 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9CC4C6D8F38B49198774F4D3D9652D95, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:27,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:27,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A6197E416AF04ED8A839C3D630E76C56
2016-05-17 21:29:27,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A6197E416AF04ED8A839C3D630E76C56 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:27,937 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:27,937 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:27,938 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A6197E416AF04ED8A839C3D630E76C56 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:27,938 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:27,938 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:27,938 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:27,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A6197E416AF04ED8A839C3D630E76C56
2016-05-17 21:29:27,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:27,942 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:27,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:27,943 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520567603_0001_1_01_000000_0
2016-05-17 21:29:27,943 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520567603_0001_1_00_000000_0_10039, runDuration: 50171, ]
2016-05-17 21:29:27,944 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520567603_0001_1_00_000000_0_10039] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520567944, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:27,944 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A6197E416AF04ED8A839C3D630E76C56
2016-05-17 21:29:27,945 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:27,945 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [A6197E416AF04ED8A839C3D630E76C56] #7 in 2ms
2016-05-17 21:29:27,945 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A6197E416AF04ED8A839C3D630E76C56] #7, status:false, isInterrupted:false
2016-05-17 21:29:27,946 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:27,947 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:27,947 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: EB8736F2AB694002BD1B995DBF595BF0
2016-05-17 21:29:27,947 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_RESULTS"], id: 9CC4C6D8F38B49198774F4D3D9652D95
2016-05-17 21:29:27,948 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_RESULTS"]
2016-05-17 21:29:27,948 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] EB8736F2AB694002BD1B995DBF595BF0, for 1 inputs
2016-05-17 21:29:27,949 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:27,949 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:27,949 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:27,949 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:27,950 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:27,950 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A6197E416AF04ED8A839C3D630E76C56
2016-05-17 21:29:27,950 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DBB52415DED3404EB2A1C730C15BFF76, all 1 inputs ready in: 00:00:00.002
2016-05-17 21:29:27,951 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:27,954 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:27,956 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@3c4539af; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@64ce2f12
2016-05-17 21:29:27,959 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:27,959 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:27,959 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520567603_0001_1_01_000000_0
2016-05-17 21:29:27,960 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520567603_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:27,960 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520567603_0001_1_01_000000_0
2016-05-17 21:29:27,960 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DBB52415DED3404EB2A1C730C15BFF76, mem on close (mb), free: 155, total: 359, max: 672
2016-05-17 21:29:27,960 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A6197E416AF04ED8A839C3D630E76C56
2016-05-17 21:29:27,961 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:27,962 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:27,962 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:27,962 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:27,962 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:27,962 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:27 Completed running task attempt: attempt_1463520567603_0001_1_01_000000_0
2016-05-17 21:29:27,964 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520567603_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:27,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520567952, Process_Duration=8, Process_End_Time=1463520567960, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:27,965 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:27,965 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:27,965 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DBB52415DED3404EB2A1C730C15BFF76, taskAttemptId=attempt_1463520567603_0001_1_01_000000_0, startTime=1463520567904, finishTime=1463520567964, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520567952, Process_Duration=8, Process_End_Time=1463520567960, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:27,965 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:27,965 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520567603_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:27,965 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520567603_0001_00_000002
2016-05-17 21:29:27,966 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:27,966 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520567603_0001_1_01_000000_0
2016-05-17 21:29:27,967 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:27,967 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:TASK_FINISHED]: vertexName=DBB52415DED3404EB2A1C730C15BFF76, taskId=task_1463520567603_0001_1_01_000000, startTime=1463520567904, finishTime=1463520567966, timeTaken=62, status=SUCCEEDED, successfulAttemptID=attempt_1463520567603_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520567952, Process_Duration=8, Process_End_Time=1463520567960, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:27,967 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520567603_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:27,967 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76] : 1
2016-05-17 21:29:27,967 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76]
2016-05-17 21:29:27,969 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:27,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:VERTEX_FINISHED]: vertexName=DBB52415DED3404EB2A1C730C15BFF76, vertexId=vertex_1463520567603_0001_1_01, initRequestedTime=1463520567788, initedTime=1463520567795, startRequestedTime=1463520567808, startedTime=1463520567808, finishTime=1463520567967, timeTaken=159, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520567952, Process_Duration=8, Process_End_Time=1463520567960, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520567904, firstTasksToStart=[ task_1463520567603_0001_1_01_000000 ], lastTaskFinishTime=1463520567964, lastTasksToFinish=[ task_1463520567603_0001_1_01_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520567603_0001_1_01_000000 ], longestDurationTasks=[ task_1463520567603_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:27,969 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:27,970 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520567603_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:27,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520567603_0001_00_000002, stoppedTime=1463520567970, exitStatus=0
2016-05-17 21:29:27,970 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520567603_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:27,970 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:27,970 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520567603_0001_1
2016-05-17 21:29:27,971 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:27,971 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520567603_0001_1
2016-05-17 21:29:27,971 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 9CC4C6D8F38B49198774F4D3D9652D95 for vertex: vertex_1463520567603_0001_1_01 [DBB52415DED3404EB2A1C730C15BFF76]
2016-05-17 21:29:27,976 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520567603_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:27,976 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520567603_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:27,976 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520567603_0001_00_000002, Exiting
2016-05-17 21:29:27,977 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520567603_0001_00_000002
2016-05-17 21:29:27,977 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520567603_0001_00_000002
2016-05-17 21:29:27,977 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520567603_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:27,977 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520567603_0001_00_000002 completed successfully
2016-05-17 21:29:27,980 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520567603_0001_1_00 [A6197E416AF04ED8A839C3D630E76C56]
2016-05-17 21:29:27,982 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520567603_0001_1, queueSize=0
2016-05-17 21:29:27,982 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:27,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520567603_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520567603_0001_1, startTime=1463520567785, finishTime=1463520567980, timeTaken=195, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=345, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=753926144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041135826, Process_Duration=10, Process_End_Time=2927041135836, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:27,983 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520567603_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:27,983 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520567603_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:27,983 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520567603_0001_00_000002
2016-05-17 21:29:27,983 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520567603_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:27 Completed Dag: dag_1463520567603_0001_1
2016-05-17 21:29:27,984 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520567603_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:27,984 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520567603_0001_1_post
2016-05-17 21:29:27,984 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:27,984 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:27,993 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_UNKNOWN_RESULTS, applicationId=application_1463520567603_0001
2016-05-17 21:29:27,996 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:27,996 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:27,997 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:27,997 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:27,997 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:27,997 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:27,998 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:27,998 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:27,998 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:27,998 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:27,998 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:27,999 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:27,999 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:28,005 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/52712520A8C3455D9480755A1075DBC3/CA084EFEB4CD48379DE655DBDADA7216/.tez/application_1463520567603_0001
2016-05-17 21:29:28,005 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_RESULTS/_temporary
2016-05-17 21:29:28,006 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.430
2016-05-17 21:29:28,079 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_RESULTS
2016-05-17 21:29:28,103 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:28,103 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:28,157 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-05-17 21:29:28,173 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.038
2016-05-17 21:29:28,174 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:28,174 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:28,174 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:28,191 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:28,198 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_ALL
2016-05-17 21:29:28,247 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:28,247 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:28,247 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_ALL"]
2016-05-17 21:29:28,248 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:28,248 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:28,248 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:28,249 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-05-17 21:29:28,249 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:28,264 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:28,270 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:28,270 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:28,271 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:28,280 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/74CF0FB293BB4235876A84241CEAC809/A4F987FDBA0B42ABB1D0AE6A1A5E4606/.tez/application_1463520568270_0001 doesn't exist and is created
2016-05-17 21:29:28,297 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:28,297 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:28,298 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/74CF0FB293BB4235876A84241CEAC809/A4F987FDBA0B42ABB1D0AE6A1A5E4606/.tez/application_1463520568270_0001
2016-05-17 21:29:28,301 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520568270_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:28,311 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:28,312 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:28,314 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:28,314 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:28,314 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/74CF0FB293BB4235876A84241CEAC809/A4F987FDBA0B42ABB1D0AE6A1A5E4606/.tez/application_1463520568270_0001/localmode-log-dir
2016-05-17 21:29:28,315 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/74CF0FB293BB4235876A84241CEAC809/A4F987FDBA0B42ABB1D0AE6A1A5E4606/.tez/application_1463520568270_0001/localmode-log-dir/history.txt.appattempt_1463520568270_0001_000000, maxErrors=10
2016-05-17 21:29:28,315 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:28,316 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520568270_0001_000000, appSubmitTime=1463520568301, launchTime=1463520568301
2016-05-17 21:29:28,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:28,320 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:28,321 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:28,321 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:28,321 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:38257
2016-05-17 21:29:28,324 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:28,327 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520568270_0001_000000, startTime=1463520568327
2016-05-17 21:29:28,327 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:28,327 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:28,397 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:28,397 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:28,397 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/-1_UNKNOWN_ALL, applicationId=application_1463520568270_0001, dagName=[74CF0FB293BB4235876A84241CEAC809/A4F987FDBA0B42ABB1D0AE6A1A5E4606] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-05-17 21:29:28,413 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [74CF0FB293BB4235876A84241CEAC809/A4F987FDBA0B42ABB1D0AE6A1A5E4606] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-05-17 21:29:28,415 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520568270_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/74CF0FB293BB4235876A84241CEAC809/A4F987FDBA0B42ABB1D0AE6A1A5E4606/.tez/application_1463520568270_0001/localmode-log-dir/dag_1463520568270_0001_1.dot
2016-05-17 21:29:28,416 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/74CF0FB293BB4235876A84241CEAC809/A4F987FDBA0B42ABB1D0AE6A1A5E4606/.tez/application_1463520568270_0001/localmode-log-dir/dag_1463520568270_0001_1-tez-dag.pb.txt
2016-05-17 21:29:28,436 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520568270_0001_1
2016-05-17 21:29:28,436 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:28,436 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [74CF0FB293BB4235876A84241CEAC809/A4F987FDBA0B42ABB1D0AE6A1A5E4606] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-05-17 21:29:28 Running Dag: dag_1463520568270_0001_1
2016-05-17 21:29:28,450 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520568270_0001_1, submitTime=1463520568413
2016-05-17 21:29:28,450 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:28,451 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BB866E990F7840F88753FB61F6566BB0
2016-05-17 21:29:28,451 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:28,451 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:28,452 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520568270_0001_1, initTime=1463520568450
2016-05-17 21:29:28,452 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520568270_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:28,452 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:DAG_STARTED]: dagID=dag_1463520568270_0001_1, startTime=1463520568452
2016-05-17 21:29:28,452 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:28,452 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:28,454 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520568270_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:28,454 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A262B492697B4FD7B2D3ADDC457D695D : {25C35B37B5CE4AB2ABE338A89C8287EB={InputName=25C35B37B5CE4AB2ABE338A89C8287EB}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:28,454 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 25C35B37B5CE4AB2ABE338A89C8287EB, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:28,454 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D]
2016-05-17 21:29:28,454 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D]
2016-05-17 21:29:28,454 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D]
2016-05-17 21:29:28,455 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D]. Starting root input initializers: 1
2016-05-17 21:29:28,453 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/-1_UNKNOWN_ALL, applicationId=application_1463520568270_0001, dagName=[74CF0FB293BB4235876A84241CEAC809/A4F987FDBA0B42ABB1D0AE6A1A5E4606] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-05-17 21:29:28,456 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:28,456 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0]
2016-05-17 21:29:28,457 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 25C35B37B5CE4AB2ABE338A89C8287EB on vertex vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D]
2016-05-17 21:29:28,459 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:28,460 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0]
2016-05-17 21:29:28,459 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:28,461 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0]
2016-05-17 21:29:28,461 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0]
2016-05-17 21:29:28,461 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B3B96439109C4DE68141F6D14093DB80, vertexId=vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:28,461 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 25C35B37B5CE4AB2ABE338A89C8287EB asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:28,463 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:28,463 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:28,462 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B3B96439109C4DE68141F6D14093DB80, vertexId=vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0]
2016-05-17 21:29:28,465 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:28,466 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 25C35B37B5CE4AB2ABE338A89C8287EB on vertex vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D]
2016-05-17 21:29:28,467 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B3B96439109C4DE68141F6D14093DB80, vertexName=BB866E990F7840F88753FB61F6566BB0, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:28,467 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B3B96439109C4DE68141F6D14093DB80, vertexId=vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0]
2016-05-17 21:29:28,468 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:28,468 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BB866E990F7840F88753FB61F6566BB0, vertexId=vertex_1463520568270_0001_1_01, initRequestedTime=1463520568456, initedTime=1463520568468, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:28,468 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:28,469 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:28,469 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {25C35B37B5CE4AB2ABE338A89C8287EB=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:28,469 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D] parallelism set to 1
2016-05-17 21:29:28,469 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:28,470 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520568270_0001_1_00, eventCount=1
2016-05-17 21:29:28,470 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A262B492697B4FD7B2D3ADDC457D695D, vertexId=vertex_1463520568270_0001_1_00, initRequestedTime=1463520568454, initedTime=1463520568470, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:28,470 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D] with distanceFromRoot: 0
2016-05-17 21:29:28,470 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:28,471 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:28,471 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520568270_0001, with dag id: dag_1463520568270_0001_1
2016-05-17 21:29:28,471 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A262B492697B4FD7B2D3ADDC457D695D
2016-05-17 21:29:28,472 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D]
2016-05-17 21:29:28,473 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520568270_0001_1_00, startRequestedTime=1463520568471, startedTime=1463520568471
2016-05-17 21:29:28,473 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:28,473 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:28,474 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:28,474 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:TASK_STARTED]: vertexName=A262B492697B4FD7B2D3ADDC457D695D, taskId=task_1463520568270_0001_1_00_000000, scheduledTime=1463520568473, launchTime=1463520568473
2016-05-17 21:29:28,474 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520568270_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:28,474 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520568270_0001_1_00 for vertex: vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0] numStartedSources: 1 numSources: 1
2016-05-17 21:29:28,474 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0] with distanceFromRoot: 1
2016-05-17 21:29:28,476 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D]
2016-05-17 21:29:28,477 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A262B492697B4FD7B2D3ADDC457D695D in vertex: BB866E990F7840F88753FB61F6566BB0
2016-05-17 21:29:28,477 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BB866E990F7840F88753FB61F6566BB0 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:28,478 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520568270_0001_1_01, startRequestedTime=1463520568477, startedTime=1463520568477
2016-05-17 21:29:28,478 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:28,478 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:28,479 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [74CF0FB293BB4235876A84241CEAC809/A4F987FDBA0B42ABB1D0AE6A1A5E4606] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL, VertexName: A262B492697B4FD7B2D3ADDC457D695D, VertexParallelism: 1, TaskAttemptID:attempt_1463520568270_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=25C35B37B5CE4AB2ABE338A89C8287EB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BB866E990F7840F88753FB61F6566BB0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:28,479 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520568270_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:28,479 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:28,480 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:28,481 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:28,481 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:28,482 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520568270_0001_00_000001
2016-05-17 21:29:28,482 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568270_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:28,484 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520568270_0001_00_000001
2016-05-17 21:29:28,484 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:28,485 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568270_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:28,485 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:28,485 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520568270_0001_00_000001, launchTime=1463520568485
2016-05-17 21:29:28,486 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520568270_0001_00_000001
2016-05-17 21:29:28,486 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520568270_0001_1_00_000000_0] to container: [container_1463520568270_0001_00_000001]
2016-05-17 21:29:28,486 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568270_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:28,487 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520568270_0001_1_00_000000_0] started. Is using containerId: [container_1463520568270_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:28,487 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A262B492697B4FD7B2D3ADDC457D695D, taskAttemptId=attempt_1463520568270_0001_1_00_000000_0, startTime=1463520568487, containerId=container_1463520568270_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520568270_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:28,487 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:28,487 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520568270_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:28,488 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520568270_0001_00_000001 given task: attempt_1463520568270_0001_1_00_000000_0
2016-05-17 21:29:28,488 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520568270_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:28,488 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520568270_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520568270_0001_1_00_000000_0
2016-05-17 21:29:28 Starting to run new task attempt: attempt_1463520568270_0001_1_00_000000_0
2016-05-17 21:29:28,489 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520568270_0001_1_00_000000_0
2016-05-17 21:29:28,489 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:28,490 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:28,490 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:28,490 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:28,490 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:28,491 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:28,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [74CF0FB293BB4235876A84241CEAC809/A4F987FDBA0B42ABB1D0AE6A1A5E4606] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL, VertexName: A262B492697B4FD7B2D3ADDC457D695D, VertexParallelism: 1, TaskAttemptID:attempt_1463520568270_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=25C35B37B5CE4AB2ABE338A89C8287EB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BB866E990F7840F88753FB61F6566BB0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:28,491 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:28,492 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520568270_0001_1_00_000000_0
2016-05-17 21:29:28,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:28,492 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:28,493 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=25C35B37B5CE4AB2ABE338A89C8287EB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:28,493 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:28,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 25C35B37B5CE4AB2ABE338A89C8287EB
2016-05-17 21:29:28,494 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 25C35B37B5CE4AB2ABE338A89C8287EB
2016-05-17 21:29:28,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:28,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BB866E990F7840F88753FB61F6566BB0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:28,496 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:28,496 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BB866E990F7840F88753FB61F6566BB0
2016-05-17 21:29:28,497 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:28,498 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BB866E990F7840F88753FB61F6566BB0
2016-05-17 21:29:28,498 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:28,501 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:28,501 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 25C35B37B5CE4AB2ABE338A89C8287EB
2016-05-17 21:29:28,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 25C35B37B5CE4AB2ABE338A89C8287EB
2016-05-17 21:29:28,503 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BB866E990F7840F88753FB61F6566BB0
2016-05-17 21:29:28,503 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520568270_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:28,506 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A262B492697B4FD7B2D3ADDC457D695D, ordinal: 0
2016-05-17 21:29:28,506 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A262B492697B4FD7B2D3ADDC457D695D, mem on start (mb), free: 265, total: 359, max: 672
2016-05-17 21:29:28,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:28,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:28,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:28,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:28,507 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:28,507 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:28,507 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:28,507 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:28,507 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:28,508 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:28,508 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 25C35B37B5CE4AB2ABE338A89C8287EB, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:28,508 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BB866E990F7840F88753FB61F6566BB0, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:28,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:28,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 25C35B37B5CE4AB2ABE338A89C8287EB
2016-05-17 21:29:28,508 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:28,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 25C35B37B5CE4AB2ABE338A89C8287EB
2016-05-17 21:29:28,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 25C35B37B5CE4AB2ABE338A89C8287EB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:28,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:28,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:28,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:28,510 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520568270_0001_1_00_000000_0
2016-05-17 21:29:28,510 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:28,511 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@648fb56b
2016-05-17 21:29:28,512 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@648fb56b
2016-05-17 21:29:28,513 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:28,513 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:28,514 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:28,514 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 25C35B37B5CE4AB2ABE338A89C8287EB
2016-05-17 21:29:28,515 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 4A13B97515CD4E7CA3F291ED0ACE7120
2016-05-17 21:29:28,515 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 4A13B97515CD4E7CA3F291ED0ACE7120
2016-05-17 21:29:28,517 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:28,519 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:28,519 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@41ce0390; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@3c4c04b0; comparator=cascading.tuple.hadoop.util.TupleComparator@22c7ddbd; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:28,520 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:28,520 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:28,520 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:28,533 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:28,534 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:28,534 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:28,534 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:28,534 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:28,535 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:28,535 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:28,535 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A262B492697B4FD7B2D3ADDC457D695D, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:28,538 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520568270_0001_1_00_000000_0
2016-05-17 21:29:28,538 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A262B492697B4FD7B2D3ADDC457D695D, mem on close (mb), free: 162, total: 359, max: 672
2016-05-17 21:29:28,538 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:28,538 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:28,538 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:28,538 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:28,542 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:28 Completed running task attempt: attempt_1463520568270_0001_1_00_000000_0
2016-05-17 21:29:28,547 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520568270_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:28,548 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520568270_0001_1_00, eventCount=1
2016-05-17 21:29:28,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520568535, Process_Duration=3, Process_End_Time=1463520568538, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:28,548 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:28,548 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:28,548 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A262B492697B4FD7B2D3ADDC457D695D, taskAttemptId=attempt_1463520568270_0001_1_00_000000_0, startTime=1463520568487, finishTime=1463520568548, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520568535, Process_Duration=3, Process_End_Time=1463520568538, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:28,549 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520568270_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:28,549 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520568270_0001_1_00_000000_0
2016-05-17 21:29:28,549 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:28,549 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520568270_0001_00_000001
2016-05-17 21:29:28,550 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:TASK_FINISHED]: vertexName=A262B492697B4FD7B2D3ADDC457D695D, taskId=task_1463520568270_0001_1_00_000000, startTime=1463520568487, finishTime=1463520568549, timeTaken=62, status=SUCCEEDED, successfulAttemptID=attempt_1463520568270_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520568535, Process_Duration=3, Process_End_Time=1463520568538, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:28,550 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520568270_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:28,550 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D] : 1
2016-05-17 21:29:28,550 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D]
2016-05-17 21:29:28,551 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:28,552 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:28,552 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:VERTEX_FINISHED]: vertexName=A262B492697B4FD7B2D3ADDC457D695D, vertexId=vertex_1463520568270_0001_1_00, initRequestedTime=1463520568454, initedTime=1463520568470, startRequestedTime=1463520568471, startedTime=1463520568471, finishTime=1463520568550, timeTaken=79, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520568535, Process_Duration=3, Process_End_Time=1463520568538, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520568487, firstTasksToStart=[ task_1463520568270_0001_1_00_000000 ], lastTaskFinishTime=1463520568548, lastTasksToFinish=[ task_1463520568270_0001_1_00_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520568270_0001_1_00_000000 ], longestDurationTasks=[ task_1463520568270_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:28,553 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:28,553 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:28,553 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568270_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:28,553 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0] attempt: attempt_1463520568270_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:28,553 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BB866E990F7840F88753FB61F6566BB0
2016-05-17 21:29:28,555 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0]
2016-05-17 21:29:28,555 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:28,555 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520568270_0001_00_000001, stoppedTime=1463520568555, exitStatus=0
2016-05-17 21:29:28,555 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568270_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:28,556 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:28,556 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:TASK_STARTED]: vertexName=BB866E990F7840F88753FB61F6566BB0, taskId=task_1463520568270_0001_1_01_000000, scheduledTime=1463520568556, launchTime=1463520568556
2016-05-17 21:29:28,556 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:28,557 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520568270_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:28,557 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0]
2016-05-17 21:29:28,557 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520568270_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:28,557 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [74CF0FB293BB4235876A84241CEAC809/A4F987FDBA0B42ABB1D0AE6A1A5E4606] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL, VertexName: BB866E990F7840F88753FB61F6566BB0, VertexParallelism: 1, TaskAttemptID:attempt_1463520568270_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A262B492697B4FD7B2D3ADDC457D695D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B3B96439109C4DE68141F6D14093DB80, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:28,558 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520568270_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:28,558 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568270_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:28,558 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:28,560 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520568270_0001_00_000002
2016-05-17 21:29:28,560 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568270_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:28,561 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520568270_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:28,562 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520568270_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:28,562 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520568270_0001_00_000001, Exiting
2016-05-17 21:29:28,562 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520568270_0001_00_000001
2016-05-17 21:29:28,562 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520568270_0001_00_000001
2016-05-17 21:29:28,562 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520568270_0001_00_000002
2016-05-17 21:29:28,565 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520568270_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:28,566 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:28,567 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520568270_0001_00_000001 completed successfully
2016-05-17 21:29:28,567 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568270_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:28,570 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568270_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:28,570 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:28,570 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520568270_0001_00_000002, launchTime=1463520568570
2016-05-17 21:29:28,571 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520568270_0001_00_000002
2016-05-17 21:29:28,572 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520568270_0001_1_01_000000_0] to container: [container_1463520568270_0001_00_000002]
2016-05-17 21:29:28,572 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568270_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:28,572 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520568270_0001_00_000002 given task: attempt_1463520568270_0001_1_01_000000_0
2016-05-17 21:29:28,573 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520568270_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520568270_0001_1_01_000000_0
2016-05-17 21:29:28,573 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520568270_0001_1_01_000000_0] started. Is using containerId: [container_1463520568270_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:28,573 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BB866E990F7840F88753FB61F6566BB0, taskAttemptId=attempt_1463520568270_0001_1_01_000000_0, startTime=1463520568572, containerId=container_1463520568270_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520568270_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:28,573 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:28,573 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520568270_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:28,573 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520568270_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:28 Starting to run new task attempt: attempt_1463520568270_0001_1_01_000000_0
2016-05-17 21:29:28,574 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520568270_0001_1_01_000000_0
2016-05-17 21:29:28,574 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:28,574 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:28,574 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:28,575 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:28,575 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:28,575 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:28,575 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [74CF0FB293BB4235876A84241CEAC809/A4F987FDBA0B42ABB1D0AE6A1A5E4606] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL, VertexName: BB866E990F7840F88753FB61F6566BB0, VertexParallelism: 1, TaskAttemptID:attempt_1463520568270_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A262B492697B4FD7B2D3ADDC457D695D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B3B96439109C4DE68141F6D14093DB80, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:28,575 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:28,577 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520568270_0001_1_01_000000_0
2016-05-17 21:29:28,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:28,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:28,578 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520568270_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:28,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A262B492697B4FD7B2D3ADDC457D695D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:28,580 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B3B96439109C4DE68141F6D14093DB80, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:28,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:28,580 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:28,581 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A262B492697B4FD7B2D3ADDC457D695D
2016-05-17 21:29:28,582 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:28,583 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:28,583 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A262B492697B4FD7B2D3ADDC457D695D
2016-05-17 21:29:28,583 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A262B492697B4FD7B2D3ADDC457D695D
2016-05-17 21:29:28,583 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:28,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:28,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B3B96439109C4DE68141F6D14093DB80
2016-05-17 21:29:28,586 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:28,586 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B3B96439109C4DE68141F6D14093DB80
2016-05-17 21:29:28,593 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BB866E990F7840F88753FB61F6566BB0, ordinal: 1
2016-05-17 21:29:28,594 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BB866E990F7840F88753FB61F6566BB0, mem on start (mb), free: 155, total: 359, max: 672
2016-05-17 21:29:28,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:28,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:28,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:28,599 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:28,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B3B96439109C4DE68141F6D14093DB80
2016-05-17 21:29:28,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:28,599 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:28,600 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:28,600 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:28,600 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:28,600 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:28,601 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:28,601 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A262B492697B4FD7B2D3ADDC457D695D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:28,601 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B3B96439109C4DE68141F6D14093DB80, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:28,601 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:28,601 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A262B492697B4FD7B2D3ADDC457D695D
2016-05-17 21:29:28,602 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:28,602 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:28,602 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A262B492697B4FD7B2D3ADDC457D695D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:28,603 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:28,603 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:28,603 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:28,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A262B492697B4FD7B2D3ADDC457D695D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:28,604 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:28,604 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A262B492697B4FD7B2D3ADDC457D695D
2016-05-17 21:29:28,604 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:28,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:28,607 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520568270_0001_1_01_000000_0
2016-05-17 21:29:28,610 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520568270_0001_1_00_000000_0_10045, runDuration: 48840, ]
2016-05-17 21:29:28,611 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:28,612 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520568270_0001_1_00_000000_0_10045] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520568612, TimeTaken=2, Rate=0.09 MB/s
2016-05-17 21:29:28,612 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A262B492697B4FD7B2D3ADDC457D695D
2016-05-17 21:29:28,612 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 4A13B97515CD4E7CA3F291ED0ACE7120
2016-05-17 21:29:28,613 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_ALL"], id: B3B96439109C4DE68141F6D14093DB80
2016-05-17 21:29:28,613 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:28,613 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [A262B492697B4FD7B2D3ADDC457D695D] #8 in 3ms
2016-05-17 21:29:28,613 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_ALL"]
2016-05-17 21:29:28,614 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 4A13B97515CD4E7CA3F291ED0ACE7120, for 1 inputs
2016-05-17 21:29:28,614 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A262B492697B4FD7B2D3ADDC457D695D] #8, status:false, isInterrupted:false
2016-05-17 21:29:28,615 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:28,616 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:28,616 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:28,616 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:28,617 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:28,617 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:28,618 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A262B492697B4FD7B2D3ADDC457D695D
2016-05-17 21:29:28,618 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:28,618 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BB866E990F7840F88753FB61F6566BB0, all 1 inputs ready in: 00:00:00.004
2016-05-17 21:29:28,620 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:28,622 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@55534f31; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@507445ba
2016-05-17 21:29:28,625 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:28,626 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:28,626 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520568270_0001_1_01_000000_0
2016-05-17 21:29:28,626 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520568270_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:28,627 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520568270_0001_1_01_000000_0
2016-05-17 21:29:28,627 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BB866E990F7840F88753FB61F6566BB0, mem on close (mb), free: 150, total: 359, max: 672
2016-05-17 21:29:28,627 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A262B492697B4FD7B2D3ADDC457D695D
2016-05-17 21:29:28,628 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:28,629 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:28,629 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:28,629 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:28,629 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:28,629 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:28 Completed running task attempt: attempt_1463520568270_0001_1_01_000000_0
2016-05-17 21:29:28,630 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520568270_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:28,632 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BB866E990F7840F88753FB61F6566BB0, taskAttemptId=attempt_1463520568270_0001_1_01_000000_0, startTime=1463520568572, finishTime=1463520568632, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520568618, Process_Duration=9, Process_End_Time=1463520568627, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:28,632 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520568270_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:28,633 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520568270_0001_1_01_000000_0
2016-05-17 21:29:28,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520568618, Process_Duration=9, Process_End_Time=1463520568627, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:28,633 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:TASK_FINISHED]: vertexName=BB866E990F7840F88753FB61F6566BB0, taskId=task_1463520568270_0001_1_01_000000, startTime=1463520568572, finishTime=1463520568633, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1463520568270_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520568618, Process_Duration=9, Process_End_Time=1463520568627, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:28,633 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:28,634 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:28,634 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:28,634 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:28,633 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520568270_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:28,634 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0] : 1
2016-05-17 21:29:28,634 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:28,634 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520568270_0001_00_000002
2016-05-17 21:29:28,634 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0]
2016-05-17 21:29:28,635 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:28,635 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:VERTEX_FINISHED]: vertexName=BB866E990F7840F88753FB61F6566BB0, vertexId=vertex_1463520568270_0001_1_01, initRequestedTime=1463520568456, initedTime=1463520568468, startRequestedTime=1463520568477, startedTime=1463520568477, finishTime=1463520568634, timeTaken=157, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520568618, Process_Duration=9, Process_End_Time=1463520568627, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520568572, firstTasksToStart=[ task_1463520568270_0001_1_01_000000 ], lastTaskFinishTime=1463520568632, lastTasksToFinish=[ task_1463520568270_0001_1_01_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520568270_0001_1_01_000000 ], longestDurationTasks=[ task_1463520568270_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:28,636 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:28,636 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568270_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:28,636 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520568270_0001_00_000002, stoppedTime=1463520568636, exitStatus=0
2016-05-17 21:29:28,636 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:28,636 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568270_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:28,636 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:28,636 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520568270_0001_1
2016-05-17 21:29:28,637 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520568270_0001_1
2016-05-17 21:29:28,637 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520568270_0001_1_00 [A262B492697B4FD7B2D3ADDC457D695D]
2016-05-17 21:29:28,637 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: B3B96439109C4DE68141F6D14093DB80 for vertex: vertex_1463520568270_0001_1_01 [BB866E990F7840F88753FB61F6566BB0]
2016-05-17 21:29:28,645 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520568270_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:28,646 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520568270_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:28,646 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520568270_0001_00_000002, Exiting
2016-05-17 21:29:28,646 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520568270_0001_00_000002
2016-05-17 21:29:28,646 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520568270_0001_00_000002
2016-05-17 21:29:28,648 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520568270_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:28,648 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520568270_0001_00_000002 completed successfully
2016-05-17 21:29:28,651 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520568270_0001_1, queueSize=0
2016-05-17 21:29:28,651 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:28,651 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568270_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520568270_0001_1, startTime=1463520568452, finishTime=1463520568649, timeTaken=197, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=451, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=753926144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041137153, Process_Duration=12, Process_End_Time=2927041137165, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:28,651 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520568270_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:28,651 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520568270_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:28,652 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520568270_0001_00_000002
2016-05-17 21:29:28,652 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568270_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:28 Completed Dag: dag_1463520568270_0001_1
2016-05-17 21:29:28,652 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520568270_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:28,652 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520568270_0001_1_post
2016-05-17 21:29:28,652 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:28,653 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:28,658 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/-1_UNKNOWN_ALL, applicationId=application_1463520568270_0001
2016-05-17 21:29:28,661 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:28,661 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:28,661 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:28,661 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:28,661 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:28,662 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:28,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:28,662 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:28,662 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:28,662 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:28,663 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:28,663 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:28,663 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:28,666 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/74CF0FB293BB4235876A84241CEAC809/A4F987FDBA0B42ABB1D0AE6A1A5E4606/.tez/application_1463520568270_0001
2016-05-17 21:29:28,667 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_ALL/_temporary
2016-05-17 21:29:28,667 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.420
2016-05-17 21:29:28,737 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_ALL
2016-05-17 21:29:28,769 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:28,769 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:28,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-17 21:29:28,834 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.038
2016-05-17 21:29:28,835 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:28,835 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:28,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:28,853 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:28,853 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:28,858 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:28,865 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_REPLACE
2016-05-17 21:29:28,910 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:28,911 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:28,911 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_REPLACE"]
2016-05-17 21:29:28,911 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:28,911 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:28,911 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:28,912 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...atform/-1_UNKNOWN_REPLACE
2016-05-17 21:29:28,912 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:28,924 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:28,928 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:28,929 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:28,929 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:28,938 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A98FD45765884A74A547E9A6962267E5/FFEB799A97DA427D9A3F7D527500798F/.tez/application_1463520568928_0001 doesn't exist and is created
2016-05-17 21:29:28,954 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:28,954 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:28,955 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A98FD45765884A74A547E9A6962267E5/FFEB799A97DA427D9A3F7D527500798F/.tez/application_1463520568928_0001
2016-05-17 21:29:28,958 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520568928_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:28,974 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:28,975 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:28,977 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:28,977 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:28,977 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A98FD45765884A74A547E9A6962267E5/FFEB799A97DA427D9A3F7D527500798F/.tez/application_1463520568928_0001/localmode-log-dir
2016-05-17 21:29:28,978 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A98FD45765884A74A547E9A6962267E5/FFEB799A97DA427D9A3F7D527500798F/.tez/application_1463520568928_0001/localmode-log-dir/history.txt.appattempt_1463520568928_0001_000000, maxErrors=10
2016-05-17 21:29:28,978 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:28,979 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520568928_0001_000000, appSubmitTime=1463520568958, launchTime=1463520568958
2016-05-17 21:29:28,981 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:28,982 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:28,983 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:28,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:28,985 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:44268
2016-05-17 21:29:28,987 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:28,990 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520568928_0001_000000, startTime=1463520568990
2016-05-17 21:29:28,990 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:28,990 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:29,054 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:29,055 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:29,055 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_REPLACE, applicationId=application_1463520568928_0001, dagName=[A98FD45765884A74A547E9A6962267E5/FFEB799A97DA427D9A3F7D527500798F] every/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-05-17 21:29:29,070 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A98FD45765884A74A547E9A6962267E5/FFEB799A97DA427D9A3F7D527500798F] every/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-05-17 21:29:29,072 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520568928_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A98FD45765884A74A547E9A6962267E5/FFEB799A97DA427D9A3F7D527500798F/.tez/application_1463520568928_0001/localmode-log-dir/dag_1463520568928_0001_1.dot
2016-05-17 21:29:29,072 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A98FD45765884A74A547E9A6962267E5/FFEB799A97DA427D9A3F7D527500798F/.tez/application_1463520568928_0001/localmode-log-dir/dag_1463520568928_0001_1-tez-dag.pb.txt
2016-05-17 21:29:29,085 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520568928_0001_1
2016-05-17 21:29:29,085 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:29,085 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A98FD45765884A74A547E9A6962267E5/FFEB799A97DA427D9A3F7D527500798F] every/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-05-17 21:29:29 Running Dag: dag_1463520568928_0001_1
2016-05-17 21:29:29,099 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520568928_0001_1, submitTime=1463520569071
2016-05-17 21:29:29,099 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:29,101 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B881F85C175C4C33A018B46333098E2C
2016-05-17 21:29:29,101 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:29,101 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:29,102 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520568928_0001_1, initTime=1463520569099
2016-05-17 21:29:29,102 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520568928_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:29,103 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:DAG_STARTED]: dagID=dag_1463520568928_0001_1, startTime=1463520569102
2016-05-17 21:29:29,103 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:29,103 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:29,103 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520568928_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:29,103 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_REPLACE, applicationId=application_1463520568928_0001, dagName=[A98FD45765884A74A547E9A6962267E5/FFEB799A97DA427D9A3F7D527500798F] every/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-05-17 21:29:29,103 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D31ABBBE5A85486C8F086398BAE4999F : {DB9513877BDE4A199983F5BDDBD5589E={InputName=DB9513877BDE4A199983F5BDDBD5589E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:29,104 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: DB9513877BDE4A199983F5BDDBD5589E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:29,104 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F]
2016-05-17 21:29:29,104 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F]
2016-05-17 21:29:29,104 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F]
2016-05-17 21:29:29,104 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F]. Starting root input initializers: 1
2016-05-17 21:29:29,105 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:29,105 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C]
2016-05-17 21:29:29,107 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: DB9513877BDE4A199983F5BDDBD5589E on vertex vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F]
2016-05-17 21:29:29,108 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:29,108 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C]
2016-05-17 21:29:29,108 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C]
2016-05-17 21:29:29,108 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C]
2016-05-17 21:29:29,109 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=219B6E22F14E4EAF9291475BA5E713E8, vertexId=vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:29,109 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=219B6E22F14E4EAF9291475BA5E713E8, vertexId=vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C]
2016-05-17 21:29:29,110 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:29,111 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input DB9513877BDE4A199983F5BDDBD5589E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:29,111 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=219B6E22F14E4EAF9291475BA5E713E8, vertexName=B881F85C175C4C33A018B46333098E2C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:29,112 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:29,112 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=219B6E22F14E4EAF9291475BA5E713E8, vertexId=vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C]
2016-05-17 21:29:29,112 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:29,114 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:29,114 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B881F85C175C4C33A018B46333098E2C, vertexId=vertex_1463520568928_0001_1_01, initRequestedTime=1463520569105, initedTime=1463520569113, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:29,114 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:29,115 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:29,115 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: DB9513877BDE4A199983F5BDDBD5589E on vertex vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F]
2016-05-17 21:29:29,115 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:29,116 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {DB9513877BDE4A199983F5BDDBD5589E=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:29,116 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F] parallelism set to 1
2016-05-17 21:29:29,116 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:29,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520568928_0001_1_00, eventCount=1
2016-05-17 21:29:29,117 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:29,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D31ABBBE5A85486C8F086398BAE4999F, vertexId=vertex_1463520568928_0001_1_00, initRequestedTime=1463520569103, initedTime=1463520569117, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:29,118 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F] with distanceFromRoot: 0
2016-05-17 21:29:29,118 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:29,119 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D31ABBBE5A85486C8F086398BAE4999F
2016-05-17 21:29:29,119 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F]
2016-05-17 21:29:29,119 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:29,119 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520568928_0001_1_00, startRequestedTime=1463520569118, startedTime=1463520569118
2016-05-17 21:29:29,120 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:29,120 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:29,121 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:TASK_STARTED]: vertexName=D31ABBBE5A85486C8F086398BAE4999F, taskId=task_1463520568928_0001_1_00_000000, scheduledTime=1463520569120, launchTime=1463520569120
2016-05-17 21:29:29,121 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520568928_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:29,121 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520568928_0001_1_00 for vertex: vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C] numStartedSources: 1 numSources: 1
2016-05-17 21:29:29,121 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520568928_0001, with dag id: dag_1463520568928_0001_1
2016-05-17 21:29:29,121 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C] with distanceFromRoot: 1
2016-05-17 21:29:29,124 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F]
2016-05-17 21:29:29,124 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D31ABBBE5A85486C8F086398BAE4999F in vertex: B881F85C175C4C33A018B46333098E2C
2016-05-17 21:29:29,125 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B881F85C175C4C33A018B46333098E2C with 1 source tasks and 1 pending tasks
2016-05-17 21:29:29,125 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:29,125 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520568928_0001_1_01, startRequestedTime=1463520569124, startedTime=1463520569124
2016-05-17 21:29:29,127 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:29,128 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A98FD45765884A74A547E9A6962267E5/FFEB799A97DA427D9A3F7D527500798F] every/(1/1) ...atform/-1_UNKNOWN_REPLACE, VertexName: D31ABBBE5A85486C8F086398BAE4999F, VertexParallelism: 1, TaskAttemptID:attempt_1463520568928_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DB9513877BDE4A199983F5BDDBD5589E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B881F85C175C4C33A018B46333098E2C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:29,128 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520568928_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:29,128 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:29,129 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:29,129 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:29,130 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:29,130 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520568928_0001_00_000001
2016-05-17 21:29:29,131 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568928_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:29,132 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520568928_0001_00_000001
2016-05-17 21:29:29,132 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:29,133 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:29,133 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568928_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:29,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520568928_0001_00_000001, launchTime=1463520569133
2016-05-17 21:29:29,135 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520568928_0001_00_000001
2016-05-17 21:29:29,135 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520568928_0001_1_00_000000_0] to container: [container_1463520568928_0001_00_000001]
2016-05-17 21:29:29,136 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568928_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:29,136 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520568928_0001_1_00_000000_0] started. Is using containerId: [container_1463520568928_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:29,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D31ABBBE5A85486C8F086398BAE4999F, taskAttemptId=attempt_1463520568928_0001_1_00_000000_0, startTime=1463520569136, containerId=container_1463520568928_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520568928_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:29,137 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520568928_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:29,137 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:29,137 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520568928_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:29,137 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520568928_0001_00_000001 given task: attempt_1463520568928_0001_1_00_000000_0
2016-05-17 21:29:29,137 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520568928_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520568928_0001_1_00_000000_0
2016-05-17 21:29:29 Starting to run new task attempt: attempt_1463520568928_0001_1_00_000000_0
2016-05-17 21:29:29,138 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520568928_0001_1_00_000000_0
2016-05-17 21:29:29,138 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:29,139 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:29,139 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:29,139 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:29,139 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:29,140 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:29,140 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A98FD45765884A74A547E9A6962267E5/FFEB799A97DA427D9A3F7D527500798F] every/(1/1) ...atform/-1_UNKNOWN_REPLACE, VertexName: D31ABBBE5A85486C8F086398BAE4999F, VertexParallelism: 1, TaskAttemptID:attempt_1463520568928_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DB9513877BDE4A199983F5BDDBD5589E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B881F85C175C4C33A018B46333098E2C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:29,140 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:29,141 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520568928_0001_1_00_000000_0
2016-05-17 21:29:29,141 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:29,141 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:29,142 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:29,142 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=DB9513877BDE4A199983F5BDDBD5589E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:29,144 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B881F85C175C4C33A018B46333098E2C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:29,144 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:29,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:29,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B881F85C175C4C33A018B46333098E2C
2016-05-17 21:29:29,145 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520568928_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:29,145 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:29,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: DB9513877BDE4A199983F5BDDBD5589E
2016-05-17 21:29:29,148 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: DB9513877BDE4A199983F5BDDBD5589E
2016-05-17 21:29:29,148 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:29,148 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B881F85C175C4C33A018B46333098E2C
2016-05-17 21:29:29,151 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B881F85C175C4C33A018B46333098E2C
2016-05-17 21:29:29,154 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D31ABBBE5A85486C8F086398BAE4999F, ordinal: 0
2016-05-17 21:29:29,154 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D31ABBBE5A85486C8F086398BAE4999F, mem on start (mb), free: 247, total: 347, max: 672
2016-05-17 21:29:29,154 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:29,154 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:29,154 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:29,155 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:29,155 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: DB9513877BDE4A199983F5BDDBD5589E
2016-05-17 21:29:29,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: DB9513877BDE4A199983F5BDDBD5589E
2016-05-17 21:29:29,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:29,156 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:29,156 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:29,156 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:29,157 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:29,158 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:29,158 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:29,159 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, DB9513877BDE4A199983F5BDDBD5589E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:29,159 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B881F85C175C4C33A018B46333098E2C, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:29,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:29,160 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: DB9513877BDE4A199983F5BDDBD5589E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:29,160 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:29,160 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:29,160 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: DB9513877BDE4A199983F5BDDBD5589E
2016-05-17 21:29:29,160 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:29,160 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: DB9513877BDE4A199983F5BDDBD5589E
2016-05-17 21:29:29,161 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:29,161 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520568928_0001_1_00_000000_0
2016-05-17 21:29:29,161 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:29,162 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@692066d7
2016-05-17 21:29:29,162 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@692066d7
2016-05-17 21:29:29,163 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:29,163 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:29,164 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:29,165 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: DB9513877BDE4A199983F5BDDBD5589E
2016-05-17 21:29:29,165 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 5942376DFD5F4F9280058AFDCA9A3866
2016-05-17 21:29:29,165 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 5942376DFD5F4F9280058AFDCA9A3866
2016-05-17 21:29:29,167 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:29,170 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:29,171 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@15dae2b2; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@55591f53; comparator=cascading.tuple.hadoop.util.TupleComparator@749b475a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:29,172 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:29,172 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:29,172 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:29,185 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:29,185 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:29,185 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:29,185 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:29,185 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:29,186 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:29,187 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:29,189 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D31ABBBE5A85486C8F086398BAE4999F, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:29,199 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520568928_0001_1_00_000000_0
2016-05-17 21:29:29,199 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D31ABBBE5A85486C8F086398BAE4999F, mem on close (mb), free: 144, total: 347, max: 672
2016-05-17 21:29:29,200 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:29,200 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:29,200 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:29,200 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:29,206 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:29 Completed running task attempt: attempt_1463520568928_0001_1_00_000000_0
2016-05-17 21:29:29,211 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520568928_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:29,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=364380160, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520569189, Process_Duration=10, Process_End_Time=1463520569199, Read_Duration=9, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:29,212 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:29,212 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:29,212 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520568928_0001_1_00, eventCount=1
2016-05-17 21:29:29,212 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520568928_0001_00_000001
2016-05-17 21:29:29,213 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:29,213 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D31ABBBE5A85486C8F086398BAE4999F, taskAttemptId=attempt_1463520568928_0001_1_00_000000_0, startTime=1463520569136, finishTime=1463520569213, timeTaken=77, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=364380160, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520569189, Process_Duration=10, Process_End_Time=1463520569199, Read_Duration=9, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:29,213 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520568928_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:29,213 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:29,213 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520568928_0001_1_00_000000_0
2016-05-17 21:29:29,214 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:29,214 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:TASK_FINISHED]: vertexName=D31ABBBE5A85486C8F086398BAE4999F, taskId=task_1463520568928_0001_1_00_000000, startTime=1463520569136, finishTime=1463520569214, timeTaken=78, status=SUCCEEDED, successfulAttemptID=attempt_1463520568928_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=364380160, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520569189, Process_Duration=10, Process_End_Time=1463520569199, Read_Duration=9, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:29,215 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520568928_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:29,215 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F] : 1
2016-05-17 21:29:29,216 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F]
2016-05-17 21:29:29,217 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:29,217 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:VERTEX_FINISHED]: vertexName=D31ABBBE5A85486C8F086398BAE4999F, vertexId=vertex_1463520568928_0001_1_00, initRequestedTime=1463520569103, initedTime=1463520569117, startRequestedTime=1463520569118, startedTime=1463520569118, finishTime=1463520569216, timeTaken=98, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=364380160, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520569189, Process_Duration=10, Process_End_Time=1463520569199, Read_Duration=9, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520569136, firstTasksToStart=[ task_1463520568928_0001_1_00_000000 ], lastTaskFinishTime=1463520569213, lastTasksToFinish=[ task_1463520568928_0001_1_00_000000 ], minTaskDuration=77, maxTaskDuration=77, avgTaskDuration=77.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520568928_0001_1_00_000000 ], longestDurationTasks=[ task_1463520568928_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:29,218 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:29,218 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568928_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:29,219 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520568928_0001_00_000001, stoppedTime=1463520569219, exitStatus=0
2016-05-17 21:29:29,219 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:29,219 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568928_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:29,220 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C] attempt: attempt_1463520568928_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:29,220 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B881F85C175C4C33A018B46333098E2C
2016-05-17 21:29:29,220 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C]
2016-05-17 21:29:29,221 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:29,221 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520568928_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:29,221 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:TASK_STARTED]: vertexName=B881F85C175C4C33A018B46333098E2C, taskId=task_1463520568928_0001_1_01_000000, scheduledTime=1463520569221, launchTime=1463520569221
2016-05-17 21:29:29,221 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:29,222 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520568928_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:29,222 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568928_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:29,223 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C]
2016-05-17 21:29:29,223 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A98FD45765884A74A547E9A6962267E5/FFEB799A97DA427D9A3F7D527500798F] every/(1/1) ...atform/-1_UNKNOWN_REPLACE, VertexName: B881F85C175C4C33A018B46333098E2C, VertexParallelism: 1, TaskAttemptID:attempt_1463520568928_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D31ABBBE5A85486C8F086398BAE4999F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=219B6E22F14E4EAF9291475BA5E713E8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:29,223 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520568928_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:29,224 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:29,224 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520568928_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:29,224 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520568928_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:29,224 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520568928_0001_00_000001, Exiting
2016-05-17 21:29:29,224 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520568928_0001_00_000001
2016-05-17 21:29:29,225 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520568928_0001_00_000001
2016-05-17 21:29:29,225 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520568928_0001_00_000002
2016-05-17 21:29:29,226 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568928_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:29,227 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520568928_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:29,227 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520568928_0001_00_000001 completed successfully
2016-05-17 21:29:29,228 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520568928_0001_00_000002
2016-05-17 21:29:29,228 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568928_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:29,229 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:29,232 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:29,232 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520568928_0001_00_000002, launchTime=1463520569231
2016-05-17 21:29:29,232 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568928_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:29,232 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520568928_0001_00_000002
2016-05-17 21:29:29,233 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520568928_0001_1_01_000000_0] to container: [container_1463520568928_0001_00_000002]
2016-05-17 21:29:29,234 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568928_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:29,234 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520568928_0001_00_000002 given task: attempt_1463520568928_0001_1_01_000000_0
2016-05-17 21:29:29,235 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520568928_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520568928_0001_1_01_000000_0
2016-05-17 21:29:29,234 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520568928_0001_1_01_000000_0] started. Is using containerId: [container_1463520568928_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:29,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:29,235 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B881F85C175C4C33A018B46333098E2C, taskAttemptId=attempt_1463520568928_0001_1_01_000000_0, startTime=1463520569234, containerId=container_1463520568928_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520568928_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:29 Starting to run new task attempt: attempt_1463520568928_0001_1_01_000000_0
2016-05-17 21:29:29,236 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520568928_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:29,237 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520568928_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:29,236 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520568928_0001_1_01_000000_0
2016-05-17 21:29:29,237 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:29,238 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:29,238 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:29,238 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:29,238 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:29,239 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:29,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A98FD45765884A74A547E9A6962267E5/FFEB799A97DA427D9A3F7D527500798F] every/(1/1) ...atform/-1_UNKNOWN_REPLACE, VertexName: B881F85C175C4C33A018B46333098E2C, VertexParallelism: 1, TaskAttemptID:attempt_1463520568928_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D31ABBBE5A85486C8F086398BAE4999F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=219B6E22F14E4EAF9291475BA5E713E8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:29,239 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:29,240 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520568928_0001_1_01_000000_0
2016-05-17 21:29:29,240 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:29,241 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520568928_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:29,241 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:29,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D31ABBBE5A85486C8F086398BAE4999F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:29,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=219B6E22F14E4EAF9291475BA5E713E8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:29,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:29,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:29,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 219B6E22F14E4EAF9291475BA5E713E8
2016-05-17 21:29:29,244 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:29,244 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 219B6E22F14E4EAF9291475BA5E713E8
2016-05-17 21:29:29,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:29,246 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D31ABBBE5A85486C8F086398BAE4999F
2016-05-17 21:29:29,246 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:29,247 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:29,247 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:29,248 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D31ABBBE5A85486C8F086398BAE4999F
2016-05-17 21:29:29,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D31ABBBE5A85486C8F086398BAE4999F
2016-05-17 21:29:29,253 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B881F85C175C4C33A018B46333098E2C, ordinal: 1
2016-05-17 21:29:29,253 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B881F85C175C4C33A018B46333098E2C, mem on start (mb), free: 136, total: 347, max: 672
2016-05-17 21:29:29,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:29,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:29,254 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:29,258 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:29,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 219B6E22F14E4EAF9291475BA5E713E8
2016-05-17 21:29:29,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:29,259 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:29,259 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:29,259 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:29,259 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:29,259 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:29,260 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:29,260 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 219B6E22F14E4EAF9291475BA5E713E8, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:29,260 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D31ABBBE5A85486C8F086398BAE4999F, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:29,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:29,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D31ABBBE5A85486C8F086398BAE4999F
2016-05-17 21:29:29,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D31ABBBE5A85486C8F086398BAE4999F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:29,260 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:29,261 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:29,261 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D31ABBBE5A85486C8F086398BAE4999F with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:29,261 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:29,262 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:29,262 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:29,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:29,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:29,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D31ABBBE5A85486C8F086398BAE4999F
2016-05-17 21:29:29,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:29,264 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520568928_0001_1_01_000000_0
2016-05-17 21:29:29,264 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520568928_0001_1_00_000000_0_10051, runDuration: 61470, ]
2016-05-17 21:29:29,266 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520568928_0001_1_00_000000_0_10051] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520569266, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:29,266 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D31ABBBE5A85486C8F086398BAE4999F
2016-05-17 21:29:29,266 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:29,266 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D31ABBBE5A85486C8F086398BAE4999F] #9 in 1ms
2016-05-17 21:29:29,267 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D31ABBBE5A85486C8F086398BAE4999F] #9, status:false, isInterrupted:false
2016-05-17 21:29:29,267 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:29,269 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 5942376DFD5F4F9280058AFDCA9A3866
2016-05-17 21:29:29,269 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_REPLACE"], id: 219B6E22F14E4EAF9291475BA5E713E8
2016-05-17 21:29:29,269 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_REPLACE"]
2016-05-17 21:29:29,270 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 5942376DFD5F4F9280058AFDCA9A3866, for 1 inputs
2016-05-17 21:29:29,269 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:29,271 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:29,271 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:29,271 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:29,271 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:29,272 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:29,272 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D31ABBBE5A85486C8F086398BAE4999F
2016-05-17 21:29:29,272 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:29,272 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B881F85C175C4C33A018B46333098E2C, all 1 inputs ready in: 00:00:00.002
2016-05-17 21:29:29,274 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:29,275 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@2b78ce80; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@83d4bcc
2016-05-17 21:29:29,277 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:29,279 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:29,279 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:29,280 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:29,280 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520568928_0001_1_01_000000_0
2016-05-17 21:29:29,281 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520568928_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:29,282 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520568928_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:29,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=364380160, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520569273, Process_Duration=9, Process_End_Time=1463520569282, Read_Duration=0, Tuples_Read=1]]
2016-05-17 21:29:29,287 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B881F85C175C4C33A018B46333098E2C, mem on close (mb), free: 132, total: 347, max: 672
2016-05-17 21:29:29,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=B881F85C175C4C33A018B46333098E2C, index=1
2016-05-17 21:29:29,288 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D31ABBBE5A85486C8F086398BAE4999F
2016-05-17 21:29:29,288 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:29,289 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:29,290 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:29,290 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:29,290 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B881F85C175C4C33A018B46333098E2C, taskAttemptId=attempt_1463520568928_0001_1_01_000000_0, startTime=1463520569234, finishTime=1463520569288, timeTaken=54, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=364380160, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520569273, Process_Duration=9, Process_End_Time=1463520569282, Read_Duration=0, Tuples_Read=1
2016-05-17 21:29:29,290 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520568928_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:29,290 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:29,291 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:29,291 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520568928_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:29,291 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:29,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=B881F85C175C4C33A018B46333098E2C, sourceVertex=D31ABBBE5A85486C8F086398BAE4999F
2016-05-17 21:29:29,291 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:29,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=B881F85C175C4C33A018B46333098E2C, sourceVertex=219B6E22F14E4EAF9291475BA5E713E8
2016-05-17 21:29:29,291 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:TASK_FINISHED]: vertexName=B881F85C175C4C33A018B46333098E2C, taskId=task_1463520568928_0001_1_01_000000, startTime=1463520569234, finishTime=1463520569291, timeTaken=57, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:29,292 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520568928_0001_00_000002
2016-05-17 21:29:29,292 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520568928_0001_00_000002
2016-05-17 21:29:29,293 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520568928_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:29,293 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520568928_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:29,294 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520568928_0001_00_000002 completed but with errors
2016-05-17 21:29:29,294 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:29,295 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C] : 1
2016-05-17 21:29:29,295 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C] because task failed: task_1463520568928_0001_1_01_000000
2016-05-17 21:29:29,296 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:29,297 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:29,297 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C]
2016-05-17 21:29:29,298 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=219B6E22F14E4EAF9291475BA5E713E8, vertexId=vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C]
2016-05-17 21:29:29,302 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:29,302 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:VERTEX_FINISHED]: vertexName=B881F85C175C4C33A018B46333098E2C, vertexId=vertex_1463520568928_0001_1_01, initRequestedTime=1463520569105, initedTime=1463520569113, startRequestedTime=1463520569124, startedTime=1463520569124, finishTime=1463520569302, timeTaken=178, status=FAILED, diagnostics=Task failed, taskId=task_1463520568928_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520569234, firstTasksToStart=[ task_1463520568928_0001_1_01_000000 ], lastTaskFinishTime=1463520569287, lastTasksToFinish=[ task_1463520568928_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:29,302 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:29,303 INFO container.AMContainerImpl (AMContainerImpl.java:transition(686)) - Container container_1463520568928_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:29,303 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:29,303 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520568928_0001_00_000002, stoppedTime=1463520569303, exitStatus=1
2016-05-17 21:29:29,304 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520568928_0001_00_000002 transitioned from RUNNING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:29,304 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520568928_0001_1_01_000000_0 failed: true container: container_1463520568928_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:29,305 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:29,305 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:29,305 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520568928_0001_1
2016-05-17 21:29:29,306 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520568928_0001_1_00 [D31ABBBE5A85486C8F086398BAE4999F]
2016-05-17 21:29:29,306 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 219B6E22F14E4EAF9291475BA5E713E8 for vertex: vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C]
2016-05-17 21:29:29,308 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520568928_0001_1, queueSize=0
2016-05-17 21:29:29,309 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:29,310 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520568928_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520568928_0001_1, startTime=1463520569102, finishTime=1463520569307, timeTaken=205, status=FAILED, diagnostics=Vertex failed, vertexName=B881F85C175C4C33A018B46333098E2C, vertexId=vertex_1463520568928_0001_1_01, diagnostics=[Task failed, taskId=task_1463520568928_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520568928_0001_1_01 [B881F85C175C4C33A018B46333098E2C] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=364380160, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520569189, Process_Duration=10, Process_End_Time=1463520569199, Read_Duration=9, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:29,311 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520568928_0001_1 finished with state: FAILED
2016-05-17 21:29:29,311 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520568928_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:29,311 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520568928_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:29 Completed Dag: dag_1463520568928_0001_1
2016-05-17 21:29:29,312 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520568928_0001_1, dagState=FAILED
2016-05-17 21:29:29,312 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520568928_0001_1_post
2016-05-17 21:29:29,313 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:29,313 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:29,321 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_UNKNOWN_REPLACE, applicationId=application_1463520568928_0001
2016-05-17 21:29:29,324 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:29,324 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:29,324 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:29,324 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:29,325 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:29,325 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:29,325 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:29,326 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:29,326 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:29,326 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:29,326 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:29,326 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:29,327 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:29,331 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A98FD45765884A74A547E9A6962267E5/FFEB799A97DA427D9A3F7D527500798F/.tez/application_1463520568928_0001
2016-05-17 21:29:29,331 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:29,332 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...atform/-1_UNKNOWN_REPLACE
2016-05-17 21:29:29,333 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:29,333 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_REPLACE/_temporary
2016-05-17 21:29:29,333 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.423
2016-05-17 21:29:29,334 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_REPLACE
2016-05-17 21:29:29,365 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:29,365 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:29,427 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-05-17 21:29:29,434 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-05-17 21:29:29,434 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:29,435 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:29,435 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:29,450 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:29,457 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_SWAP
2016-05-17 21:29:29,527 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:29,527 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:29,528 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_SWAP"]
2016-05-17 21:29:29,528 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:29,528 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:29,529 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:29,530 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-05-17 21:29:29,530 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:29,547 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:29,552 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:29,552 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:29,553 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:29,562 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/55080189C7EB44AB843A5C260EB61E51/DAEF06C7B63F4358B6DB80579E286C0B/.tez/application_1463520569552_0001 doesn't exist and is created
2016-05-17 21:29:29,580 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:29,581 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:29,582 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/55080189C7EB44AB843A5C260EB61E51/DAEF06C7B63F4358B6DB80579E286C0B/.tez/application_1463520569552_0001
2016-05-17 21:29:29,585 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520569552_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:29,595 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:29,595 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:29,597 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:29,598 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:29,598 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/55080189C7EB44AB843A5C260EB61E51/DAEF06C7B63F4358B6DB80579E286C0B/.tez/application_1463520569552_0001/localmode-log-dir
2016-05-17 21:29:29,599 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/55080189C7EB44AB843A5C260EB61E51/DAEF06C7B63F4358B6DB80579E286C0B/.tez/application_1463520569552_0001/localmode-log-dir/history.txt.appattempt_1463520569552_0001_000000, maxErrors=10
2016-05-17 21:29:29,599 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:29,599 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520569552_0001_000000, appSubmitTime=1463520569584, launchTime=1463520569584
2016-05-17 21:29:29,602 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:29,604 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:37395
2016-05-17 21:29:29,605 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:29,608 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:29,609 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:29,611 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:29,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520569552_0001_000000, startTime=1463520569613
2016-05-17 21:29:29,613 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:29,613 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:29,681 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:29,681 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:29,681 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_UNKNOWN_SWAP, applicationId=application_1463520569552_0001, dagName=[55080189C7EB44AB843A5C260EB61E51/DAEF06C7B63F4358B6DB80579E286C0B] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-05-17 21:29:29,697 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [55080189C7EB44AB843A5C260EB61E51/DAEF06C7B63F4358B6DB80579E286C0B] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-05-17 21:29:29,698 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520569552_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/55080189C7EB44AB843A5C260EB61E51/DAEF06C7B63F4358B6DB80579E286C0B/.tez/application_1463520569552_0001/localmode-log-dir/dag_1463520569552_0001_1.dot
2016-05-17 21:29:29,699 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/55080189C7EB44AB843A5C260EB61E51/DAEF06C7B63F4358B6DB80579E286C0B/.tez/application_1463520569552_0001/localmode-log-dir/dag_1463520569552_0001_1-tez-dag.pb.txt
2016-05-17 21:29:29,708 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520569552_0001_1
2016-05-17 21:29:29,708 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:29,709 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [55080189C7EB44AB843A5C260EB61E51/DAEF06C7B63F4358B6DB80579E286C0B] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-05-17 21:29:29 Running Dag: dag_1463520569552_0001_1
2016-05-17 21:29:29,720 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520569552_0001_1, submitTime=1463520569697
2016-05-17 21:29:29,720 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:29,721 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F6E01949DF1240D09E89911ACF98A6D0
2016-05-17 21:29:29,722 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:29,722 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:29,722 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520569552_0001_1, initTime=1463520569721
2016-05-17 21:29:29,723 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520569552_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:29,723 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:DAG_STARTED]: dagID=dag_1463520569552_0001_1, startTime=1463520569723
2016-05-17 21:29:29,724 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:29,724 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520569552_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:29,724 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F6C3C4E995A0465F9E45A9EC9554699A : {B59B27E1A84C4FD4B3A287FBF2444ADA={InputName=B59B27E1A84C4FD4B3A287FBF2444ADA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:29,724 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B59B27E1A84C4FD4B3A287FBF2444ADA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:29,724 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A]
2016-05-17 21:29:29,724 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_UNKNOWN_SWAP, applicationId=application_1463520569552_0001, dagName=[55080189C7EB44AB843A5C260EB61E51/DAEF06C7B63F4358B6DB80579E286C0B] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-05-17 21:29:29,724 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A]
2016-05-17 21:29:29,724 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A]
2016-05-17 21:29:29,725 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A]. Starting root input initializers: 1
2016-05-17 21:29:29,723 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:29,726 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:29,726 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0]
2016-05-17 21:29:29,728 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B59B27E1A84C4FD4B3A287FBF2444ADA on vertex vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A]
2016-05-17 21:29:29,729 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:29,729 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0]
2016-05-17 21:29:29,730 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0]
2016-05-17 21:29:29,730 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0]
2016-05-17 21:29:29,730 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D175A80060B34330A2BD4E78CCC254FB, vertexId=vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:29,731 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D175A80060B34330A2BD4E78CCC254FB, vertexId=vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0]
2016-05-17 21:29:29,733 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D175A80060B34330A2BD4E78CCC254FB, vertexName=F6E01949DF1240D09E89911ACF98A6D0, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:29,733 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D175A80060B34330A2BD4E78CCC254FB, vertexId=vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0]
2016-05-17 21:29:29,735 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:29,735 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F6E01949DF1240D09E89911ACF98A6D0, vertexId=vertex_1463520569552_0001_1_01, initRequestedTime=1463520569726, initedTime=1463520569735, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:29,735 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:29,736 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:29,736 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B59B27E1A84C4FD4B3A287FBF2444ADA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:29,737 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:29,737 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:29,741 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:29,741 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B59B27E1A84C4FD4B3A287FBF2444ADA on vertex vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A]
2016-05-17 21:29:29,742 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:29,742 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B59B27E1A84C4FD4B3A287FBF2444ADA=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:29,743 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A] parallelism set to 1
2016-05-17 21:29:29,743 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:29,744 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520569552_0001, with dag id: dag_1463520569552_0001_1
2016-05-17 21:29:29,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520569552_0001_1_00, eventCount=1
2016-05-17 21:29:29,745 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F6C3C4E995A0465F9E45A9EC9554699A, vertexId=vertex_1463520569552_0001_1_00, initRequestedTime=1463520569724, initedTime=1463520569745, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:29,745 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A] with distanceFromRoot: 0
2016-05-17 21:29:29,745 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:29,746 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:29,747 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F6C3C4E995A0465F9E45A9EC9554699A
2016-05-17 21:29:29,747 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A]
2016-05-17 21:29:29,748 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:29,748 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:29,748 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:29,749 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520569552_0001_1_00, startRequestedTime=1463520569747, startedTime=1463520569747
2016-05-17 21:29:29,749 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:29,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:29,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:TASK_STARTED]: vertexName=F6C3C4E995A0465F9E45A9EC9554699A, taskId=task_1463520569552_0001_1_00_000000, scheduledTime=1463520569749, launchTime=1463520569749
2016-05-17 21:29:29,750 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520569552_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:29,751 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520569552_0001_1_00 for vertex: vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0] numStartedSources: 1 numSources: 1
2016-05-17 21:29:29,751 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0] with distanceFromRoot: 1
2016-05-17 21:29:29,751 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A]
2016-05-17 21:29:29,752 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F6C3C4E995A0465F9E45A9EC9554699A in vertex: F6E01949DF1240D09E89911ACF98A6D0
2016-05-17 21:29:29,752 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F6E01949DF1240D09E89911ACF98A6D0 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:29,753 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:29,753 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520569552_0001_1_01, startRequestedTime=1463520569752, startedTime=1463520569752
2016-05-17 21:29:29,753 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:29,754 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [55080189C7EB44AB843A5C260EB61E51/DAEF06C7B63F4358B6DB80579E286C0B] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP, VertexName: F6C3C4E995A0465F9E45A9EC9554699A, VertexParallelism: 1, TaskAttemptID:attempt_1463520569552_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B59B27E1A84C4FD4B3A287FBF2444ADA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F6E01949DF1240D09E89911ACF98A6D0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:29,754 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520569552_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:29,755 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:29,756 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:29,757 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:29,757 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:29,757 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520569552_0001_00_000001
2016-05-17 21:29:29,757 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520569552_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:29,760 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520569552_0001_00_000001
2016-05-17 21:29:29,760 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:29,761 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520569552_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:29,761 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:29,761 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520569552_0001_00_000001, launchTime=1463520569761
2016-05-17 21:29:29,762 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520569552_0001_00_000001
2016-05-17 21:29:29,762 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520569552_0001_1_00_000000_0] to container: [container_1463520569552_0001_00_000001]
2016-05-17 21:29:29,763 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520569552_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:29,763 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520569552_0001_00_000001 given task: attempt_1463520569552_0001_1_00_000000_0
2016-05-17 21:29:29,763 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520569552_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520569552_0001_1_00_000000_0
2016-05-17 21:29:29,763 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520569552_0001_1_00_000000_0] started. Is using containerId: [container_1463520569552_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:29,764 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F6C3C4E995A0465F9E45A9EC9554699A, taskAttemptId=attempt_1463520569552_0001_1_00_000000_0, startTime=1463520569763, containerId=container_1463520569552_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520569552_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:29,764 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:29 Starting to run new task attempt: attempt_1463520569552_0001_1_00_000000_0
2016-05-17 21:29:29,764 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520569552_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:29,765 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520569552_0001_1_00_000000_0
2016-05-17 21:29:29,765 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:29,765 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520569552_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:29,765 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:29,765 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:29,765 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:29,765 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:29,765 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:29,766 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [55080189C7EB44AB843A5C260EB61E51/DAEF06C7B63F4358B6DB80579E286C0B] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP, VertexName: F6C3C4E995A0465F9E45A9EC9554699A, VertexParallelism: 1, TaskAttemptID:attempt_1463520569552_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B59B27E1A84C4FD4B3A287FBF2444ADA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F6E01949DF1240D09E89911ACF98A6D0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:29,766 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:29,766 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520569552_0001_1_00_000000_0
2016-05-17 21:29:29,766 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:29,766 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:29,768 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520569552_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:29,768 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:29,768 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B59B27E1A84C4FD4B3A287FBF2444ADA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:29,769 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:29,769 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B59B27E1A84C4FD4B3A287FBF2444ADA
2016-05-17 21:29:29,769 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B59B27E1A84C4FD4B3A287FBF2444ADA
2016-05-17 21:29:29,771 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:29,771 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B59B27E1A84C4FD4B3A287FBF2444ADA
2016-05-17 21:29:29,771 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:29,771 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F6E01949DF1240D09E89911ACF98A6D0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:29,771 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B59B27E1A84C4FD4B3A287FBF2444ADA
2016-05-17 21:29:29,772 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:29,772 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F6E01949DF1240D09E89911ACF98A6D0
2016-05-17 21:29:29,773 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:29,773 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F6E01949DF1240D09E89911ACF98A6D0
2016-05-17 21:29:29,774 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F6E01949DF1240D09E89911ACF98A6D0
2016-05-17 21:29:29,780 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F6C3C4E995A0465F9E45A9EC9554699A, ordinal: 0
2016-05-17 21:29:29,781 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F6C3C4E995A0465F9E45A9EC9554699A, mem on start (mb), free: 266, total: 359, max: 672
2016-05-17 21:29:29,781 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:29,781 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:29,781 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:29,781 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:29,781 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:29,781 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:29,781 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:29,781 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:29,781 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:29,782 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:29,782 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B59B27E1A84C4FD4B3A287FBF2444ADA, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:29,782 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F6E01949DF1240D09E89911ACF98A6D0, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:29,782 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:29,782 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B59B27E1A84C4FD4B3A287FBF2444ADA being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:29,783 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:29,783 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:29,783 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B59B27E1A84C4FD4B3A287FBF2444ADA
2016-05-17 21:29:29,784 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:29,784 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B59B27E1A84C4FD4B3A287FBF2444ADA
2016-05-17 21:29:29,784 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:29,784 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520569552_0001_1_00_000000_0
2016-05-17 21:29:29,785 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:29,785 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@175e9358
2016-05-17 21:29:29,787 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@175e9358
2016-05-17 21:29:29,787 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:29,787 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:29,788 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:29,789 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B59B27E1A84C4FD4B3A287FBF2444ADA
2016-05-17 21:29:29,789 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 4F6849E2464948CF9198CFF74FC7E4A2
2016-05-17 21:29:29,789 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 4F6849E2464948CF9198CFF74FC7E4A2
2016-05-17 21:29:29,790 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:29,793 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:29,793 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@122a10c5; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@7b2fed4; comparator=cascading.tuple.hadoop.util.TupleComparator@7878143e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:29,793 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:29,794 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:29,794 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:29,807 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:29,807 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:29,807 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:29,808 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:29,808 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:29,809 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:29,809 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:29,809 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F6C3C4E995A0465F9E45A9EC9554699A, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:29,812 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520569552_0001_1_00_000000_0
2016-05-17 21:29:29,812 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F6C3C4E995A0465F9E45A9EC9554699A, mem on close (mb), free: 164, total: 359, max: 672
2016-05-17 21:29:29,812 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:29,812 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:29,812 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:29,813 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:29,817 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:29 Completed running task attempt: attempt_1463520569552_0001_1_00_000000_0
2016-05-17 21:29:29,822 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520569552_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:29,822 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520569552_0001_1_00, eventCount=1
2016-05-17 21:29:29,823 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:29,823 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F6C3C4E995A0465F9E45A9EC9554699A, taskAttemptId=attempt_1463520569552_0001_1_00_000000_0, startTime=1463520569763, finishTime=1463520569823, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520569809, Process_Duration=2, Process_End_Time=1463520569811, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:29,823 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520569552_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:29,823 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520569552_0001_1_00_000000_0
2016-05-17 21:29:29,823 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:29,824 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:29,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:TASK_FINISHED]: vertexName=F6C3C4E995A0465F9E45A9EC9554699A, taskId=task_1463520569552_0001_1_00_000000, startTime=1463520569763, finishTime=1463520569823, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1463520569552_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520569809, Process_Duration=2, Process_End_Time=1463520569811, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:29,824 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520569552_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:29,824 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A] : 1
2016-05-17 21:29:29,824 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A]
2016-05-17 21:29:29,826 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:29,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:VERTEX_FINISHED]: vertexName=F6C3C4E995A0465F9E45A9EC9554699A, vertexId=vertex_1463520569552_0001_1_00, initRequestedTime=1463520569724, initedTime=1463520569745, startRequestedTime=1463520569747, startedTime=1463520569747, finishTime=1463520569825, timeTaken=78, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520569809, Process_Duration=2, Process_End_Time=1463520569811, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520569763, firstTasksToStart=[ task_1463520569552_0001_1_00_000000 ], lastTaskFinishTime=1463520569823, lastTasksToFinish=[ task_1463520569552_0001_1_00_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520569552_0001_1_00_000000 ], longestDurationTasks=[ task_1463520569552_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:29,826 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:29,827 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520569552_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:29,827 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0] attempt: attempt_1463520569552_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:29,827 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F6E01949DF1240D09E89911ACF98A6D0
2016-05-17 21:29:29,827 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0]
2016-05-17 21:29:29,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520569552_0001_00_000001, stoppedTime=1463520569827, exitStatus=0
2016-05-17 21:29:29,828 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520569552_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:29,828 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:29,828 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:29,828 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:TASK_STARTED]: vertexName=F6E01949DF1240D09E89911ACF98A6D0, taskId=task_1463520569552_0001_1_01_000000, scheduledTime=1463520569828, launchTime=1463520569828
2016-05-17 21:29:29,828 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:29,828 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520569809, Process_Duration=2, Process_End_Time=1463520569811, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:29,828 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520569552_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:29,829 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:29,829 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:29,829 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520569552_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:29,829 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0]
2016-05-17 21:29:29,830 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520569552_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:29,830 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [55080189C7EB44AB843A5C260EB61E51/DAEF06C7B63F4358B6DB80579E286C0B] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP, VertexName: F6E01949DF1240D09E89911ACF98A6D0, VertexParallelism: 1, TaskAttemptID:attempt_1463520569552_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F6C3C4E995A0465F9E45A9EC9554699A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D175A80060B34330A2BD4E78CCC254FB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:29,830 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520569552_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:29,830 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:29,830 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520569552_0001_00_000001
2016-05-17 21:29:29,831 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520569552_0001_00_000002
2016-05-17 21:29:29,831 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520569552_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:29,831 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520569552_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:29,832 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520569552_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:29,832 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520569552_0001_00_000001, Exiting
2016-05-17 21:29:29,832 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520569552_0001_00_000001
2016-05-17 21:29:29,833 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520569552_0001_00_000001
2016-05-17 21:29:29,833 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520569552_0001_00_000002
2016-05-17 21:29:29,834 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520569552_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:29,834 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520569552_0001_00_000001 completed successfully
2016-05-17 21:29:29,835 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:29,835 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520569552_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:29,837 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520569552_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:29,837 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:29,838 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520569552_0001_00_000002, launchTime=1463520569837
2016-05-17 21:29:29,839 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520569552_0001_00_000002
2016-05-17 21:29:29,839 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520569552_0001_1_01_000000_0] to container: [container_1463520569552_0001_00_000002]
2016-05-17 21:29:29,839 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520569552_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:29,839 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520569552_0001_00_000002 given task: attempt_1463520569552_0001_1_01_000000_0
2016-05-17 21:29:29,840 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520569552_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520569552_0001_1_01_000000_0
2016-05-17 21:29:29 Starting to run new task attempt: attempt_1463520569552_0001_1_01_000000_0
2016-05-17 21:29:29,840 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520569552_0001_1_01_000000_0
2016-05-17 21:29:29,840 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520569552_0001_1_01_000000_0] started. Is using containerId: [container_1463520569552_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:29,840 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:29,840 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F6E01949DF1240D09E89911ACF98A6D0, taskAttemptId=attempt_1463520569552_0001_1_01_000000_0, startTime=1463520569840, containerId=container_1463520569552_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520569552_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:29,840 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:29,841 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:29,841 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:29,841 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:29,841 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:29,842 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:29,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [55080189C7EB44AB843A5C260EB61E51/DAEF06C7B63F4358B6DB80579E286C0B] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP, VertexName: F6E01949DF1240D09E89911ACF98A6D0, VertexParallelism: 1, TaskAttemptID:attempt_1463520569552_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F6C3C4E995A0465F9E45A9EC9554699A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D175A80060B34330A2BD4E78CCC254FB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:29,841 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520569552_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:29,842 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520569552_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:29,843 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:29,843 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520569552_0001_1_01_000000_0
2016-05-17 21:29:29,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:29,847 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:29,848 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520569552_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:29,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F6C3C4E995A0465F9E45A9EC9554699A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:29,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:29,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:29,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F6C3C4E995A0465F9E45A9EC9554699A
2016-05-17 21:29:29,850 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:29,850 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:29,850 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F6C3C4E995A0465F9E45A9EC9554699A
2016-05-17 21:29:29,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F6C3C4E995A0465F9E45A9EC9554699A
2016-05-17 21:29:29,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D175A80060B34330A2BD4E78CCC254FB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:29,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:29,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D175A80060B34330A2BD4E78CCC254FB
2016-05-17 21:29:29,851 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:29,851 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D175A80060B34330A2BD4E78CCC254FB
2016-05-17 21:29:29,851 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:29,862 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F6E01949DF1240D09E89911ACF98A6D0, ordinal: 1
2016-05-17 21:29:29,862 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F6E01949DF1240D09E89911ACF98A6D0, mem on start (mb), free: 157, total: 359, max: 672
2016-05-17 21:29:29,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:29,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:29,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:29,866 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:29,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D175A80060B34330A2BD4E78CCC254FB
2016-05-17 21:29:29,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:29,866 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:29,867 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:29,867 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:29,868 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:29,868 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:29,868 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:29,869 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F6C3C4E995A0465F9E45A9EC9554699A, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:29,869 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D175A80060B34330A2BD4E78CCC254FB, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:29,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:29,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F6C3C4E995A0465F9E45A9EC9554699A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:29,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F6C3C4E995A0465F9E45A9EC9554699A
2016-05-17 21:29:29,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:29,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:29,870 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:29,870 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:29,871 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F6C3C4E995A0465F9E45A9EC9554699A with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:29,871 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:29,871 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:29,871 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:29,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F6C3C4E995A0465F9E45A9EC9554699A
2016-05-17 21:29:29,873 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:29,873 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520569552_0001_1_01_000000_0
2016-05-17 21:29:29,874 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520569552_0001_1_00_000000_0_10057, runDuration: 46611, ]
2016-05-17 21:29:29,875 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520569552_0001_1_00_000000_0_10057] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520569875, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:29,875 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F6C3C4E995A0465F9E45A9EC9554699A
2016-05-17 21:29:29,876 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:29,876 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [F6C3C4E995A0465F9E45A9EC9554699A] #10 in 2ms
2016-05-17 21:29:29,876 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F6C3C4E995A0465F9E45A9EC9554699A] #10, status:false, isInterrupted:false
2016-05-17 21:29:29,877 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:29,878 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:29,878 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:29,878 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:29,878 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:29,878 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:29,878 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 4F6849E2464948CF9198CFF74FC7E4A2
2016-05-17 21:29:29,879 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_SWAP"], id: D175A80060B34330A2BD4E78CCC254FB
2016-05-17 21:29:29,879 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:29,879 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F6C3C4E995A0465F9E45A9EC9554699A
2016-05-17 21:29:29,879 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:29,879 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_SWAP"]
2016-05-17 21:29:29,879 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 4F6849E2464948CF9198CFF74FC7E4A2, for 1 inputs
2016-05-17 21:29:29,879 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F6E01949DF1240D09E89911ACF98A6D0, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:29,882 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:29,884 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@4d2432a5; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@3791634
2016-05-17 21:29:29,887 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:29,887 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:29,888 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520569552_0001_1_01_000000_0
2016-05-17 21:29:29,888 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520569552_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:29,888 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520569552_0001_1_01_000000_0
2016-05-17 21:29:29,888 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F6E01949DF1240D09E89911ACF98A6D0, mem on close (mb), free: 152, total: 359, max: 672
2016-05-17 21:29:29,889 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F6C3C4E995A0465F9E45A9EC9554699A
2016-05-17 21:29:29,889 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:29,890 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:29,890 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:29,890 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:29,890 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:29,891 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:29 Completed running task attempt: attempt_1463520569552_0001_1_01_000000_0
2016-05-17 21:29:29,892 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520569552_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:29,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:29,895 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F6E01949DF1240D09E89911ACF98A6D0, taskAttemptId=attempt_1463520569552_0001_1_01_000000_0, startTime=1463520569840, finishTime=1463520569894, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520569879, Process_Duration=9, Process_End_Time=1463520569888, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:29,895 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520569552_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:29,895 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:29,895 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520569552_0001_1_01_000000_0
2016-05-17 21:29:29,896 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:TASK_FINISHED]: vertexName=F6E01949DF1240D09E89911ACF98A6D0, taskId=task_1463520569552_0001_1_01_000000, startTime=1463520569840, finishTime=1463520569896, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1463520569552_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520569879, Process_Duration=9, Process_End_Time=1463520569888, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:29,897 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520569552_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:29,897 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0] : 1
2016-05-17 21:29:29,897 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0]
2016-05-17 21:29:29,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520569879, Process_Duration=9, Process_End_Time=1463520569888, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:29,898 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:29,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:29,898 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:29,898 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520569552_0001_00_000002
2016-05-17 21:29:29,899 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:29,899 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:VERTEX_FINISHED]: vertexName=F6E01949DF1240D09E89911ACF98A6D0, vertexId=vertex_1463520569552_0001_1_01, initRequestedTime=1463520569726, initedTime=1463520569735, startRequestedTime=1463520569752, startedTime=1463520569752, finishTime=1463520569897, timeTaken=145, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520569879, Process_Duration=9, Process_End_Time=1463520569888, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520569840, firstTasksToStart=[ task_1463520569552_0001_1_01_000000 ], lastTaskFinishTime=1463520569894, lastTasksToFinish=[ task_1463520569552_0001_1_01_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520569552_0001_1_01_000000 ], longestDurationTasks=[ task_1463520569552_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:29,899 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:29,899 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520569552_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:29,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520569552_0001_00_000002, stoppedTime=1463520569899, exitStatus=0
2016-05-17 21:29:29,900 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:29,900 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520569552_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:29,900 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:29,900 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520569552_0001_1
2016-05-17 21:29:29,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520569552_0001_1
2016-05-17 21:29:29,900 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D175A80060B34330A2BD4E78CCC254FB for vertex: vertex_1463520569552_0001_1_01 [F6E01949DF1240D09E89911ACF98A6D0]
2016-05-17 21:29:29,909 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520569552_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:29,909 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520569552_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:29,909 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520569552_0001_00_000002, Exiting
2016-05-17 21:29:29,909 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520569552_0001_00_000002
2016-05-17 21:29:29,909 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520569552_0001_00_000002
2016-05-17 21:29:29,913 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520569552_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:29,913 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520569552_0001_00_000002 completed successfully
2016-05-17 21:29:29,913 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520569552_0001_1_00 [F6C3C4E995A0465F9E45A9EC9554699A]
2016-05-17 21:29:29,916 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520569552_0001_1, queueSize=0
2016-05-17 21:29:29,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:29,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520569552_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520569552_0001_1, startTime=1463520569723, finishTime=1463520569914, timeTaken=191, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=920, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=753926144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041139688, Process_Duration=11, Process_End_Time=2927041139699, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:29,917 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520569552_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:29,917 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520569552_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:29,918 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520569552_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:29,919 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520569552_0001_00_000002
2016-05-17 21:29:29 Completed Dag: dag_1463520569552_0001_1
2016-05-17 21:29:29,920 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520569552_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:29,920 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520569552_0001_1_post
2016-05-17 21:29:29,920 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:29,920 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:29,926 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_UNKNOWN_SWAP, applicationId=application_1463520569552_0001
2016-05-17 21:29:29,929 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:29,929 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:29,929 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:29,929 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:29,929 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:29,929 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:29,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:29,930 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:29,930 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:29,930 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:29,933 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:29,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:29,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:29,938 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/55080189C7EB44AB843A5C260EB61E51/DAEF06C7B63F4358B6DB80579E286C0B/.tez/application_1463520569552_0001
2016-05-17 21:29:29,939 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_SWAP/_temporary
2016-05-17 21:29:29,940 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.412
2016-05-17 21:29:30,031 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_SWAP
2016-05-17 21:29:30,063 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:30,063 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:30,121 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 21:29:30,135 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-17 21:29:30,136 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:30,136 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:30,137 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:30,154 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:30,160 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_-1
2016-05-17 21:29:30,217 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:30,217 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:30,218 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_-1"]
2016-05-17 21:29:30,218 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:30,218 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:30,219 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:30,219 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...mblyplatform/-1_VALUES_-1
2016-05-17 21:29:30,219 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:30,231 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:30,235 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:30,235 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:30,235 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:30,244 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/62FA7EC607384D04BEEDCD1796090AE0/B352A91F467F46CFBD49C7E61772DBE0/.tez/application_1463520570235_0001 doesn't exist and is created
2016-05-17 21:29:30,259 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:30,259 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:30,260 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/62FA7EC607384D04BEEDCD1796090AE0/B352A91F467F46CFBD49C7E61772DBE0/.tez/application_1463520570235_0001
2016-05-17 21:29:30,263 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520570235_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:30,277 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:30,278 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:30,281 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:30,281 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:30,281 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/62FA7EC607384D04BEEDCD1796090AE0/B352A91F467F46CFBD49C7E61772DBE0/.tez/application_1463520570235_0001/localmode-log-dir
2016-05-17 21:29:30,281 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/62FA7EC607384D04BEEDCD1796090AE0/B352A91F467F46CFBD49C7E61772DBE0/.tez/application_1463520570235_0001/localmode-log-dir/history.txt.appattempt_1463520570235_0001_000000, maxErrors=10
2016-05-17 21:29:30,281 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:30,282 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520570235_0001_000000, appSubmitTime=1463520570262, launchTime=1463520570262
2016-05-17 21:29:30,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:30,286 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:30,287 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:30,287 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:30,288 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53199
2016-05-17 21:29:30,291 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:30,294 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520570235_0001_000000, startTime=1463520570294
2016-05-17 21:29:30,294 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:30,294 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:30,359 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:30,360 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:30,360 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_VALUES_-1, applicationId=application_1463520570235_0001, dagName=[62FA7EC607384D04BEEDCD1796090AE0/B352A91F467F46CFBD49C7E61772DBE0] every/(1/1) ...mblyplatform/-1_VALUES_-1
2016-05-17 21:29:30,374 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [62FA7EC607384D04BEEDCD1796090AE0/B352A91F467F46CFBD49C7E61772DBE0] every/(1/1) ...mblyplatform/-1_VALUES_-1
2016-05-17 21:29:30,376 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520570235_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/62FA7EC607384D04BEEDCD1796090AE0/B352A91F467F46CFBD49C7E61772DBE0/.tez/application_1463520570235_0001/localmode-log-dir/dag_1463520570235_0001_1.dot
2016-05-17 21:29:30,376 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/62FA7EC607384D04BEEDCD1796090AE0/B352A91F467F46CFBD49C7E61772DBE0/.tez/application_1463520570235_0001/localmode-log-dir/dag_1463520570235_0001_1-tez-dag.pb.txt
2016-05-17 21:29:30,384 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520570235_0001_1
2016-05-17 21:29:30,384 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:30,384 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [62FA7EC607384D04BEEDCD1796090AE0/B352A91F467F46CFBD49C7E61772DBE0] every/(1/1) ...mblyplatform/-1_VALUES_-1
2016-05-17 21:29:30 Running Dag: dag_1463520570235_0001_1
2016-05-17 21:29:30,395 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520570235_0001_1, submitTime=1463520570375
2016-05-17 21:29:30,395 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:30,395 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DAC7108033964B02A8AC013371A3DB10
2016-05-17 21:29:30,396 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:30,396 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520570235_0001_1, initTime=1463520570395
2016-05-17 21:29:30,396 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:30,396 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520570235_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:30,397 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:DAG_STARTED]: dagID=dag_1463520570235_0001_1, startTime=1463520570397
2016-05-17 21:29:30,397 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:30,398 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_VALUES_-1, applicationId=application_1463520570235_0001, dagName=[62FA7EC607384D04BEEDCD1796090AE0/B352A91F467F46CFBD49C7E61772DBE0] every/(1/1) ...mblyplatform/-1_VALUES_-1
2016-05-17 21:29:30,397 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:30,398 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520570235_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:30,398 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C7CE61F1041C4AEBBC6822FE3B402CAD : {F448434185794A548B4E1C5EFCCB678C={InputName=F448434185794A548B4E1C5EFCCB678C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:30,398 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F448434185794A548B4E1C5EFCCB678C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:30,398 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD]
2016-05-17 21:29:30,398 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD]
2016-05-17 21:29:30,398 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD]
2016-05-17 21:29:30,399 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD]. Starting root input initializers: 1
2016-05-17 21:29:30,399 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:30,399 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10]
2016-05-17 21:29:30,399 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F448434185794A548B4E1C5EFCCB678C on vertex vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD]
2016-05-17 21:29:30,402 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:30,402 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10]
2016-05-17 21:29:30,402 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10]
2016-05-17 21:29:30,402 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10]
2016-05-17 21:29:30,402 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1B5E81575B794EE4BF321D4B2C902673, vertexId=vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:30,402 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1B5E81575B794EE4BF321D4B2C902673, vertexId=vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10]
2016-05-17 21:29:30,403 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:30,403 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F448434185794A548B4E1C5EFCCB678C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:30,404 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1B5E81575B794EE4BF321D4B2C902673, vertexName=DAC7108033964B02A8AC013371A3DB10, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:30,404 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:30,404 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1B5E81575B794EE4BF321D4B2C902673, vertexId=vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10]
2016-05-17 21:29:30,404 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:30,405 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:30,406 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DAC7108033964B02A8AC013371A3DB10, vertexId=vertex_1463520570235_0001_1_01, initRequestedTime=1463520570399, initedTime=1463520570405, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:30,406 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:30,406 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:30,406 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F448434185794A548B4E1C5EFCCB678C on vertex vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD]
2016-05-17 21:29:30,406 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:30,407 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F448434185794A548B4E1C5EFCCB678C=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:30,407 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD] parallelism set to 1
2016-05-17 21:29:30,407 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:30,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520570235_0001_1_00, eventCount=1
2016-05-17 21:29:30,408 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:30,408 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C7CE61F1041C4AEBBC6822FE3B402CAD, vertexId=vertex_1463520570235_0001_1_00, initRequestedTime=1463520570398, initedTime=1463520570408, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:30,408 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD] with distanceFromRoot: 0
2016-05-17 21:29:30,409 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:30,409 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C7CE61F1041C4AEBBC6822FE3B402CAD
2016-05-17 21:29:30,409 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD]
2016-05-17 21:29:30,409 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:30,410 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520570235_0001_1_00, startRequestedTime=1463520570409, startedTime=1463520570409
2016-05-17 21:29:30,410 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:30,410 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:30,410 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:TASK_STARTED]: vertexName=C7CE61F1041C4AEBBC6822FE3B402CAD, taskId=task_1463520570235_0001_1_00_000000, scheduledTime=1463520570410, launchTime=1463520570410
2016-05-17 21:29:30,410 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520570235_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:30,411 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520570235_0001_1_00 for vertex: vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10] numStartedSources: 1 numSources: 1
2016-05-17 21:29:30,411 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10] with distanceFromRoot: 1
2016-05-17 21:29:30,411 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD]
2016-05-17 21:29:30,411 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C7CE61F1041C4AEBBC6822FE3B402CAD in vertex: DAC7108033964B02A8AC013371A3DB10
2016-05-17 21:29:30,411 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DAC7108033964B02A8AC013371A3DB10 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:30,411 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520570235_0001_1_01, startRequestedTime=1463520570411, startedTime=1463520570411
2016-05-17 21:29:30,412 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:30,412 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:30,412 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [62FA7EC607384D04BEEDCD1796090AE0/B352A91F467F46CFBD49C7E61772DBE0] every/(1/1) ...mblyplatform/-1_VALUES_-1, VertexName: C7CE61F1041C4AEBBC6822FE3B402CAD, VertexParallelism: 1, TaskAttemptID:attempt_1463520570235_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F448434185794A548B4E1C5EFCCB678C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DAC7108033964B02A8AC013371A3DB10, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:30,412 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520570235_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:30,412 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:30,413 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520570235_0001, with dag id: dag_1463520570235_0001_1
2016-05-17 21:29:30,413 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:30,414 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:30,414 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:30,414 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520570235_0001_00_000001
2016-05-17 21:29:30,415 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570235_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:30,416 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520570235_0001_00_000001
2016-05-17 21:29:30,417 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:30,417 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570235_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:30,418 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520570235_0001_00_000001, launchTime=1463520570417
2016-05-17 21:29:30,417 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:30,419 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520570235_0001_00_000001
2016-05-17 21:29:30,419 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520570235_0001_1_00_000000_0] to container: [container_1463520570235_0001_00_000001]
2016-05-17 21:29:30,419 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570235_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:30,419 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520570235_0001_00_000001 given task: attempt_1463520570235_0001_1_00_000000_0
2016-05-17 21:29:30,420 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520570235_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520570235_0001_1_00_000000_0
2016-05-17 21:29:30 Starting to run new task attempt: attempt_1463520570235_0001_1_00_000000_0
2016-05-17 21:29:30,420 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520570235_0001_1_00_000000_0] started. Is using containerId: [container_1463520570235_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:30,420 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520570235_0001_1_00_000000_0
2016-05-17 21:29:30,420 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:30,420 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:30,421 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:30,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C7CE61F1041C4AEBBC6822FE3B402CAD, taskAttemptId=attempt_1463520570235_0001_1_00_000000_0, startTime=1463520570420, containerId=container_1463520570235_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520570235_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:30,421 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:30,421 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520570235_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:30,421 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:30,421 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520570235_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:30,422 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:30,422 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:30,422 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [62FA7EC607384D04BEEDCD1796090AE0/B352A91F467F46CFBD49C7E61772DBE0] every/(1/1) ...mblyplatform/-1_VALUES_-1, VertexName: C7CE61F1041C4AEBBC6822FE3B402CAD, VertexParallelism: 1, TaskAttemptID:attempt_1463520570235_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F448434185794A548B4E1C5EFCCB678C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DAC7108033964B02A8AC013371A3DB10, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:30,422 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:30,423 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520570235_0001_1_00_000000_0
2016-05-17 21:29:30,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:30,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:30,423 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520570235_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:30,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F448434185794A548B4E1C5EFCCB678C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:30,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DAC7108033964B02A8AC013371A3DB10, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:30,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:30,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:30,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F448434185794A548B4E1C5EFCCB678C
2016-05-17 21:29:30,425 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F448434185794A548B4E1C5EFCCB678C
2016-05-17 21:29:30,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:30,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DAC7108033964B02A8AC013371A3DB10
2016-05-17 21:29:30,426 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:30,426 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F448434185794A548B4E1C5EFCCB678C
2016-05-17 21:29:30,426 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F448434185794A548B4E1C5EFCCB678C
2016-05-17 21:29:30,427 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:30,427 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:30,427 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DAC7108033964B02A8AC013371A3DB10
2016-05-17 21:29:30,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DAC7108033964B02A8AC013371A3DB10
2016-05-17 21:29:30,446 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C7CE61F1041C4AEBBC6822FE3B402CAD, ordinal: 0
2016-05-17 21:29:30,446 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C7CE61F1041C4AEBBC6822FE3B402CAD, mem on start (mb), free: 262, total: 358, max: 672
2016-05-17 21:29:30,446 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:30,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:30,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:30,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:30,447 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:30,448 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:30,448 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:30,448 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:30,448 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:30,449 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:30,449 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F448434185794A548B4E1C5EFCCB678C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:30,449 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DAC7108033964B02A8AC013371A3DB10, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:30,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:30,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F448434185794A548B4E1C5EFCCB678C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:30,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:30,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:30,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F448434185794A548B4E1C5EFCCB678C
2016-05-17 21:29:30,451 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:30,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F448434185794A548B4E1C5EFCCB678C
2016-05-17 21:29:30,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:30,452 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520570235_0001_1_00_000000_0
2016-05-17 21:29:30,452 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:30,453 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@53f8823d
2016-05-17 21:29:30,454 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@53f8823d
2016-05-17 21:29:30,454 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:30,454 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:30,455 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:30,456 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F448434185794A548B4E1C5EFCCB678C
2016-05-17 21:29:30,456 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C57C3026F0B54257A6A90F4D9B247513
2016-05-17 21:29:30,456 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C57C3026F0B54257A6A90F4D9B247513
2016-05-17 21:29:30,457 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:30,460 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:30,460 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@70e4c940; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@3c58c2a7; comparator=cascading.tuple.hadoop.util.TupleComparator@675380e4; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:30,460 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:30,461 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:30,461 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:30,474 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:30,474 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:30,474 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:30,475 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:30,475 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:30,476 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:30,476 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:30,476 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C7CE61F1041C4AEBBC6822FE3B402CAD, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:30,481 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520570235_0001_1_00_000000_0
2016-05-17 21:29:30,481 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C7CE61F1041C4AEBBC6822FE3B402CAD, mem on close (mb), free: 160, total: 358, max: 672
2016-05-17 21:29:30,481 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:30,481 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:30,482 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:30,482 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:30,485 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:30 Completed running task attempt: attempt_1463520570235_0001_1_00_000000_0
2016-05-17 21:29:30,489 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520570235_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:30,490 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520570477, Process_Duration=4, Process_End_Time=1463520570481, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:30,490 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520570235_0001_1_00, eventCount=1
2016-05-17 21:29:30,490 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:30,490 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:30,490 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520570235_0001_00_000001
2016-05-17 21:29:30,491 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C7CE61F1041C4AEBBC6822FE3B402CAD, taskAttemptId=attempt_1463520570235_0001_1_00_000000_0, startTime=1463520570420, finishTime=1463520570490, timeTaken=70, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520570477, Process_Duration=4, Process_End_Time=1463520570481, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:30,491 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:30,491 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520570235_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:30,491 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520570235_0001_1_00_000000_0
2016-05-17 21:29:30,491 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:30,492 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:30,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:TASK_FINISHED]: vertexName=C7CE61F1041C4AEBBC6822FE3B402CAD, taskId=task_1463520570235_0001_1_00_000000, startTime=1463520570420, finishTime=1463520570491, timeTaken=71, status=SUCCEEDED, successfulAttemptID=attempt_1463520570235_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520570477, Process_Duration=4, Process_End_Time=1463520570481, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:30,492 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520570235_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:30,492 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD] : 1
2016-05-17 21:29:30,492 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD]
2016-05-17 21:29:30,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:30,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:VERTEX_FINISHED]: vertexName=C7CE61F1041C4AEBBC6822FE3B402CAD, vertexId=vertex_1463520570235_0001_1_00, initRequestedTime=1463520570398, initedTime=1463520570408, startRequestedTime=1463520570409, startedTime=1463520570409, finishTime=1463520570492, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520570477, Process_Duration=4, Process_End_Time=1463520570481, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520570420, firstTasksToStart=[ task_1463520570235_0001_1_00_000000 ], lastTaskFinishTime=1463520570490, lastTasksToFinish=[ task_1463520570235_0001_1_00_000000 ], minTaskDuration=70, maxTaskDuration=70, avgTaskDuration=70.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520570235_0001_1_00_000000 ], longestDurationTasks=[ task_1463520570235_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:30,498 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:30,498 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570235_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:30,499 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10] attempt: attempt_1463520570235_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:30,499 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DAC7108033964B02A8AC013371A3DB10
2016-05-17 21:29:30,499 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10]
2016-05-17 21:29:30,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520570235_0001_00_000001, stoppedTime=1463520570500, exitStatus=0
2016-05-17 21:29:30,500 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570235_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:30,500 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:30,500 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:30,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:TASK_STARTED]: vertexName=DAC7108033964B02A8AC013371A3DB10, taskId=task_1463520570235_0001_1_01_000000, scheduledTime=1463520570501, launchTime=1463520570501
2016-05-17 21:29:30,501 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520570235_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:30,501 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520570235_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:30,502 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520570235_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:30,502 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520570235_0001_00_000001, Exiting
2016-05-17 21:29:30,502 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520570235_0001_00_000001
2016-05-17 21:29:30,502 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520570235_0001_00_000001
2016-05-17 21:29:30,501 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:30,501 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520570235_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:30,501 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10]
2016-05-17 21:29:30,505 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570235_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:30,505 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [62FA7EC607384D04BEEDCD1796090AE0/B352A91F467F46CFBD49C7E61772DBE0] every/(1/1) ...mblyplatform/-1_VALUES_-1, VertexName: DAC7108033964B02A8AC013371A3DB10, VertexParallelism: 1, TaskAttemptID:attempt_1463520570235_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C7CE61F1041C4AEBBC6822FE3B402CAD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1B5E81575B794EE4BF321D4B2C902673, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:30,506 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520570235_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:30,506 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:30,506 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520570235_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:30,506 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520570235_0001_00_000001 completed successfully
2016-05-17 21:29:30,506 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520570235_0001_00_000002
2016-05-17 21:29:30,507 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570235_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:30,507 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570235_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:30,508 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520570235_0001_00_000002
2016-05-17 21:29:30,509 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:30,512 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570235_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:30,512 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:30,513 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520570235_0001_00_000002, launchTime=1463520570512
2016-05-17 21:29:30,513 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520570235_0001_00_000002
2016-05-17 21:29:30,514 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520570235_0001_1_01_000000_0] to container: [container_1463520570235_0001_00_000002]
2016-05-17 21:29:30,514 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570235_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:30,514 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520570235_0001_00_000002 given task: attempt_1463520570235_0001_1_01_000000_0
2016-05-17 21:29:30,515 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520570235_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520570235_0001_1_01_000000_0
2016-05-17 21:29:30,515 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520570235_0001_1_01_000000_0] started. Is using containerId: [container_1463520570235_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:30,515 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DAC7108033964B02A8AC013371A3DB10, taskAttemptId=attempt_1463520570235_0001_1_01_000000_0, startTime=1463520570514, containerId=container_1463520570235_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520570235_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:30 Starting to run new task attempt: attempt_1463520570235_0001_1_01_000000_0
2016-05-17 21:29:30,515 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520570235_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:30,515 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520570235_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:30,516 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:30,515 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520570235_0001_1_01_000000_0
2016-05-17 21:29:30,516 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:30,516 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:30,516 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:30,516 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:30,516 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:30,517 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:30,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [62FA7EC607384D04BEEDCD1796090AE0/B352A91F467F46CFBD49C7E61772DBE0] every/(1/1) ...mblyplatform/-1_VALUES_-1, VertexName: DAC7108033964B02A8AC013371A3DB10, VertexParallelism: 1, TaskAttemptID:attempt_1463520570235_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C7CE61F1041C4AEBBC6822FE3B402CAD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1B5E81575B794EE4BF321D4B2C902673, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:30,517 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:30,519 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520570235_0001_1_01_000000_0
2016-05-17 21:29:30,519 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:30,519 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:30,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:30,522 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520570235_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:30,525 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:30,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C7CE61F1041C4AEBBC6822FE3B402CAD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:30,526 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:30,526 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:30,526 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:30,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C7CE61F1041C4AEBBC6822FE3B402CAD
2016-05-17 21:29:30,528 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1B5E81575B794EE4BF321D4B2C902673, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:30,528 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:30,528 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1B5E81575B794EE4BF321D4B2C902673
2016-05-17 21:29:30,528 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:30,530 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1B5E81575B794EE4BF321D4B2C902673
2016-05-17 21:29:30,530 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:30,530 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:30,531 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C7CE61F1041C4AEBBC6822FE3B402CAD
2016-05-17 21:29:30,534 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C7CE61F1041C4AEBBC6822FE3B402CAD
2016-05-17 21:29:30,537 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DAC7108033964B02A8AC013371A3DB10, ordinal: 1
2016-05-17 21:29:30,537 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DAC7108033964B02A8AC013371A3DB10, mem on start (mb), free: 153, total: 358, max: 672
2016-05-17 21:29:30,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:30,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:30,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:30,544 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:30,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1B5E81575B794EE4BF321D4B2C902673
2016-05-17 21:29:30,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:30,544 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:30,545 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:30,545 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:30,545 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:30,545 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:30,546 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:30,546 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1B5E81575B794EE4BF321D4B2C902673, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:30,546 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C7CE61F1041C4AEBBC6822FE3B402CAD, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:30,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:30,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C7CE61F1041C4AEBBC6822FE3B402CAD
2016-05-17 21:29:30,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C7CE61F1041C4AEBBC6822FE3B402CAD being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:30,547 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:30,547 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:30,548 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C7CE61F1041C4AEBBC6822FE3B402CAD with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:30,548 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:30,548 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:30,548 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:30,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:30,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:30,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C7CE61F1041C4AEBBC6822FE3B402CAD
2016-05-17 21:29:30,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:30,551 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520570235_0001_1_01_000000_0
2016-05-17 21:29:30,551 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520570235_0001_1_00_000000_0_10063, runDuration: 61827, ]
2016-05-17 21:29:30,553 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520570235_0001_1_00_000000_0_10063] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520570553, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:30,553 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C7CE61F1041C4AEBBC6822FE3B402CAD
2016-05-17 21:29:30,554 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:30,555 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [C7CE61F1041C4AEBBC6822FE3B402CAD] #11 in 3ms
2016-05-17 21:29:30,555 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C7CE61F1041C4AEBBC6822FE3B402CAD] #11, status:false, isInterrupted:false
2016-05-17 21:29:30,556 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:30,557 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:30,557 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:30,557 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:30,557 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:30,557 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:30,558 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:30,558 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C7CE61F1041C4AEBBC6822FE3B402CAD
2016-05-17 21:29:30,558 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:30,558 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C57C3026F0B54257A6A90F4D9B247513
2016-05-17 21:29:30,559 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_-1"], id: 1B5E81575B794EE4BF321D4B2C902673
2016-05-17 21:29:30,559 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_-1"]
2016-05-17 21:29:30,560 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C57C3026F0B54257A6A90F4D9B247513, for 1 inputs
2016-05-17 21:29:30,560 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DAC7108033964B02A8AC013371A3DB10, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:30,561 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:30,563 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@29e7371e; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@ad3a810
2016-05-17 21:29:30,563 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:30,565 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:30,567 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:30,567 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:30,568 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520570235_0001_1_01_000000_0
2016-05-17 21:29:30,568 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520570235_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:30,570 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520570235_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:30,576 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520570560, Process_Duration=10, Process_End_Time=1463520570570, Read_Duration=0, Tuples_Read=1]]
2016-05-17 21:29:30,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:30,576 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DAC7108033964B02A8AC013371A3DB10, taskAttemptId=attempt_1463520570235_0001_1_01_000000_0, startTime=1463520570514, finishTime=1463520570576, timeTaken=62, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520570560, Process_Duration=10, Process_End_Time=1463520570570, Read_Duration=0, Tuples_Read=1
2016-05-17 21:29:30,578 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520570235_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:30,578 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520570235_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:30,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:TASK_FINISHED]: vertexName=DAC7108033964B02A8AC013371A3DB10, taskId=task_1463520570235_0001_1_01_000000, startTime=1463520570514, finishTime=1463520570578, timeTaken=64, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:30,579 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520570235_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:30,579 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10] : 1
2016-05-17 21:29:30,579 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10] because task failed: task_1463520570235_0001_1_01_000000
2016-05-17 21:29:30,579 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:30,579 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:30,579 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10]
2016-05-17 21:29:30,579 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=1B5E81575B794EE4BF321D4B2C902673, vertexId=vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10]
2016-05-17 21:29:30,581 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:30,583 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:30,583 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:30,577 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DAC7108033964B02A8AC013371A3DB10, mem on close (mb), free: 147, total: 358, max: 672
2016-05-17 21:29:30,583 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:VERTEX_FINISHED]: vertexName=DAC7108033964B02A8AC013371A3DB10, vertexId=vertex_1463520570235_0001_1_01, initRequestedTime=1463520570399, initedTime=1463520570405, startRequestedTime=1463520570411, startedTime=1463520570411, finishTime=1463520570581, timeTaken=170, status=FAILED, diagnostics=Task failed, taskId=task_1463520570235_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520570514, firstTasksToStart=[ task_1463520570235_0001_1_01_000000 ], lastTaskFinishTime=1463520570576, lastTasksToFinish=[ task_1463520570235_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:30,591 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:30,590 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=DAC7108033964B02A8AC013371A3DB10, index=1
2016-05-17 21:29:30,591 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:30,591 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:30,591 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520570235_0001_1
2016-05-17 21:29:30,591 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 1B5E81575B794EE4BF321D4B2C902673 for vertex: vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10]
2016-05-17 21:29:30,592 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C7CE61F1041C4AEBBC6822FE3B402CAD
2016-05-17 21:29:30,592 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:30,592 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520570235_0001_1_00 [C7CE61F1041C4AEBBC6822FE3B402CAD]
2016-05-17 21:29:30,594 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:30,594 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:30,594 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:30,594 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520570235_0001_1, queueSize=0
2016-05-17 21:29:30,594 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:30,595 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520570235_0001_1, startTime=1463520570397, finishTime=1463520570593, timeTaken=196, status=FAILED, diagnostics=Vertex failed, vertexName=DAC7108033964B02A8AC013371A3DB10, vertexId=vertex_1463520570235_0001_1_01, diagnostics=[Task failed, taskId=task_1463520570235_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520570235_0001_1_01 [DAC7108033964B02A8AC013371A3DB10] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520570477, Process_Duration=4, Process_End_Time=1463520570481, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:30,596 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520570235_0001_1 finished with state: FAILED
2016-05-17 21:29:30,596 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520570235_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:30,597 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:30,597 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570235_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520570235_0001_00_000002, stoppedTime=1463520570597, exitStatus=0
2016-05-17 21:29:30,597 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570235_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:30,597 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520570235_0001_1_01_000000_0 failed: true container: container_1463520570235_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:30,594 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:30,598 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:30,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=DAC7108033964B02A8AC013371A3DB10, sourceVertex=C7CE61F1041C4AEBBC6822FE3B402CAD
2016-05-17 21:29:30,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=DAC7108033964B02A8AC013371A3DB10, sourceVertex=1B5E81575B794EE4BF321D4B2C902673
2016-05-17 21:29:30,599 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520570235_0001_00_000002
2016-05-17 21:29:30,599 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520570235_0001_00_000002
2016-05-17 21:29:30,601 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520570235_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:30,601 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520570235_0001_00_000002 completed but with errors
2016-05-17 21:29:30 Completed Dag: dag_1463520570235_0001_1
2016-05-17 21:29:30,602 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520570235_0001_1, dagState=FAILED
2016-05-17 21:29:30,602 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520570235_0001_1_post
2016-05-17 21:29:30,602 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:30,602 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:30,602 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520570235_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:30,602 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570235_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:30,602 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520570235_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:30,603 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520570235_0001_00_000002
2016-05-17 21:29:30,606 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/-1_VALUES_-1, applicationId=application_1463520570235_0001
2016-05-17 21:29:30,608 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:30,608 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:30,608 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:30,608 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:30,608 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:30,609 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:30,609 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:30,609 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:30,609 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:30,610 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:30,610 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:30,610 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:30,610 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:30,615 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/62FA7EC607384D04BEEDCD1796090AE0/B352A91F467F46CFBD49C7E61772DBE0/.tez/application_1463520570235_0001
2016-05-17 21:29:30,615 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:30,616 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...mblyplatform/-1_VALUES_-1
2016-05-17 21:29:30,616 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:30,616 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_-1/_temporary
2016-05-17 21:29:30,617 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.399
2016-05-17 21:29:30,617 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_-1
2016-05-17 21:29:30,650 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:30,651 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:30,679 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 21:29:30,682 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-05-17 21:29:30,682 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:30,682 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:30,682 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_'field'
2016-05-17 21:29:30,705 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:30,705 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:30,751 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 21:29:30,758 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 21:29:30,758 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:30,758 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:30,758 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:30,772 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:30,778 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_RESULTS
2016-05-17 21:29:30,822 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:30,822 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:30,822 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_RESULTS"]
2016-05-17 21:29:30,822 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:30,823 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:30,823 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:30,823 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...latform/-1_VALUES_RESULTS
2016-05-17 21:29:30,823 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:30,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:30,840 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:30,840 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:30,840 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:30,849 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FB087F25B6A415483E3D9709C0CBB5D/AB1CB9EE09C04EEE98507937BA1B33AA/.tez/application_1463520570840_0001 doesn't exist and is created
2016-05-17 21:29:30,864 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:30,864 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:30,865 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FB087F25B6A415483E3D9709C0CBB5D/AB1CB9EE09C04EEE98507937BA1B33AA/.tez/application_1463520570840_0001
2016-05-17 21:29:30,868 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520570840_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:30,877 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:30,878 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:30,879 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:30,879 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:30,880 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FB087F25B6A415483E3D9709C0CBB5D/AB1CB9EE09C04EEE98507937BA1B33AA/.tez/application_1463520570840_0001/localmode-log-dir
2016-05-17 21:29:30,880 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FB087F25B6A415483E3D9709C0CBB5D/AB1CB9EE09C04EEE98507937BA1B33AA/.tez/application_1463520570840_0001/localmode-log-dir/history.txt.appattempt_1463520570840_0001_000000, maxErrors=10
2016-05-17 21:29:30,880 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:30,880 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520570840_0001_000000, appSubmitTime=1463520570867, launchTime=1463520570868
2016-05-17 21:29:30,887 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:45366
2016-05-17 21:29:30,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:30,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:30,891 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:30,891 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:30,892 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:30,894 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520570840_0001_000000, startTime=1463520570894
2016-05-17 21:29:30,894 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:30,894 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:30,965 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:30,965 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:30,965 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_RESULTS, applicationId=application_1463520570840_0001, dagName=[7FB087F25B6A415483E3D9709C0CBB5D/AB1CB9EE09C04EEE98507937BA1B33AA] every/(1/1) ...latform/-1_VALUES_RESULTS
2016-05-17 21:29:30,981 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7FB087F25B6A415483E3D9709C0CBB5D/AB1CB9EE09C04EEE98507937BA1B33AA] every/(1/1) ...latform/-1_VALUES_RESULTS
2016-05-17 21:29:30,982 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520570840_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FB087F25B6A415483E3D9709C0CBB5D/AB1CB9EE09C04EEE98507937BA1B33AA/.tez/application_1463520570840_0001/localmode-log-dir/dag_1463520570840_0001_1.dot
2016-05-17 21:29:30,983 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FB087F25B6A415483E3D9709C0CBB5D/AB1CB9EE09C04EEE98507937BA1B33AA/.tez/application_1463520570840_0001/localmode-log-dir/dag_1463520570840_0001_1-tez-dag.pb.txt
2016-05-17 21:29:30,990 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520570840_0001_1
2016-05-17 21:29:30,990 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:30,990 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7FB087F25B6A415483E3D9709C0CBB5D/AB1CB9EE09C04EEE98507937BA1B33AA] every/(1/1) ...latform/-1_VALUES_RESULTS
2016-05-17 21:29:30 Running Dag: dag_1463520570840_0001_1
2016-05-17 21:29:31,004 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520570840_0001_1, submitTime=1463520570981
2016-05-17 21:29:31,004 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:31,004 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D882215E3BA84A77A204223357F3C09D
2016-05-17 21:29:31,005 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:31,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:31,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520570840_0001_1, initTime=1463520571004
2016-05-17 21:29:31,006 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520570840_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:31,006 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:DAG_STARTED]: dagID=dag_1463520570840_0001_1, startTime=1463520571006
2016-05-17 21:29:31,007 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:31,007 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520570840_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:31,007 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CAB32D803EEB44F7A88ABF175C3F0829 : {A0C164E7E7824787A0B310E2A47D5AB6={InputName=A0C164E7E7824787A0B310E2A47D5AB6}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:31,007 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A0C164E7E7824787A0B310E2A47D5AB6, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:31,007 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829]
2016-05-17 21:29:31,007 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829]
2016-05-17 21:29:31,007 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829]
2016-05-17 21:29:31,007 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829]. Starting root input initializers: 1
2016-05-17 21:29:31,008 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:31,008 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:31,008 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D]
2016-05-17 21:29:31,008 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_RESULTS, applicationId=application_1463520570840_0001, dagName=[7FB087F25B6A415483E3D9709C0CBB5D/AB1CB9EE09C04EEE98507937BA1B33AA] every/(1/1) ...latform/-1_VALUES_RESULTS
2016-05-17 21:29:31,010 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:31,010 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D]
2016-05-17 21:29:31,011 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D]
2016-05-17 21:29:31,011 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D]
2016-05-17 21:29:31,011 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=313EB353AC88421999DB4BAED6340038, vertexId=vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:31,011 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=313EB353AC88421999DB4BAED6340038, vertexId=vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D]
2016-05-17 21:29:31,011 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A0C164E7E7824787A0B310E2A47D5AB6 on vertex vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829]
2016-05-17 21:29:31,013 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=313EB353AC88421999DB4BAED6340038, vertexName=D882215E3BA84A77A204223357F3C09D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:31,013 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:31,014 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=313EB353AC88421999DB4BAED6340038, vertexId=vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D]
2016-05-17 21:29:31,014 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A0C164E7E7824787A0B310E2A47D5AB6 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:31,014 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:31,014 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:31,015 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:31,015 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D882215E3BA84A77A204223357F3C09D, vertexId=vertex_1463520570840_0001_1_01, initRequestedTime=1463520571008, initedTime=1463520571015, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:31,015 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:31,016 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:31,016 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A0C164E7E7824787A0B310E2A47D5AB6 on vertex vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829]
2016-05-17 21:29:31,016 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:31,017 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A0C164E7E7824787A0B310E2A47D5AB6=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:31,017 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829] parallelism set to 1
2016-05-17 21:29:31,017 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:31,017 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520570840_0001_1_00, eventCount=1
2016-05-17 21:29:31,018 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:31,018 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CAB32D803EEB44F7A88ABF175C3F0829, vertexId=vertex_1463520570840_0001_1_00, initRequestedTime=1463520571007, initedTime=1463520571018, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:31,018 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829] with distanceFromRoot: 0
2016-05-17 21:29:31,018 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:31,019 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CAB32D803EEB44F7A88ABF175C3F0829
2016-05-17 21:29:31,019 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829]
2016-05-17 21:29:31,019 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:31,019 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520570840_0001_1_00, startRequestedTime=1463520571019, startedTime=1463520571019
2016-05-17 21:29:31,020 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:31,020 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:31,020 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:TASK_STARTED]: vertexName=CAB32D803EEB44F7A88ABF175C3F0829, taskId=task_1463520570840_0001_1_00_000000, scheduledTime=1463520571020, launchTime=1463520571020
2016-05-17 21:29:31,020 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520570840_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:31,021 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520570840_0001_1_00 for vertex: vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D] numStartedSources: 1 numSources: 1
2016-05-17 21:29:31,021 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D] with distanceFromRoot: 1
2016-05-17 21:29:31,021 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829]
2016-05-17 21:29:31,021 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CAB32D803EEB44F7A88ABF175C3F0829 in vertex: D882215E3BA84A77A204223357F3C09D
2016-05-17 21:29:31,022 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D882215E3BA84A77A204223357F3C09D with 1 source tasks and 1 pending tasks
2016-05-17 21:29:31,022 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:31,022 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520570840_0001_1_01, startRequestedTime=1463520571021, startedTime=1463520571021
2016-05-17 21:29:31,022 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:31,023 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7FB087F25B6A415483E3D9709C0CBB5D/AB1CB9EE09C04EEE98507937BA1B33AA] every/(1/1) ...latform/-1_VALUES_RESULTS, VertexName: CAB32D803EEB44F7A88ABF175C3F0829, VertexParallelism: 1, TaskAttemptID:attempt_1463520570840_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A0C164E7E7824787A0B310E2A47D5AB6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D882215E3BA84A77A204223357F3C09D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:31,023 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520570840_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:31,023 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:31,024 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:31,024 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:31,025 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:31,025 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520570840_0001_00_000001
2016-05-17 21:29:31,025 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570840_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:31,027 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520570840_0001_00_000001
2016-05-17 21:29:31,027 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:31,029 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570840_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:31,030 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:31,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520570840_0001_00_000001, launchTime=1463520571029
2016-05-17 21:29:31,031 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520570840_0001_00_000001
2016-05-17 21:29:31,032 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520570840_0001_1_00_000000_0] to container: [container_1463520570840_0001_00_000001]
2016-05-17 21:29:31,032 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570840_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:31,033 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520570840_0001_1_00_000000_0] started. Is using containerId: [container_1463520570840_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:31,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CAB32D803EEB44F7A88ABF175C3F0829, taskAttemptId=attempt_1463520570840_0001_1_00_000000_0, startTime=1463520571033, containerId=container_1463520570840_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520570840_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:31,034 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520570840_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:31,034 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520570840_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:31,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:31,035 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520570840_0001_00_000001 given task: attempt_1463520570840_0001_1_00_000000_0
2016-05-17 21:29:31,036 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520570840_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520570840_0001_1_00_000000_0
2016-05-17 21:29:31 Starting to run new task attempt: attempt_1463520570840_0001_1_00_000000_0
2016-05-17 21:29:31,036 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520570840_0001_1_00_000000_0
2016-05-17 21:29:31,037 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:31,037 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:31,037 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:31,037 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:31,037 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:31,037 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:31,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7FB087F25B6A415483E3D9709C0CBB5D/AB1CB9EE09C04EEE98507937BA1B33AA] every/(1/1) ...latform/-1_VALUES_RESULTS, VertexName: CAB32D803EEB44F7A88ABF175C3F0829, VertexParallelism: 1, TaskAttemptID:attempt_1463520570840_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A0C164E7E7824787A0B310E2A47D5AB6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D882215E3BA84A77A204223357F3C09D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:31,038 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:31,040 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520570840_0001_1_00_000000_0
2016-05-17 21:29:31,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:31,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:31,041 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520570840_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:31,043 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A0C164E7E7824787A0B310E2A47D5AB6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:31,043 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:31,044 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A0C164E7E7824787A0B310E2A47D5AB6
2016-05-17 21:29:31,044 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A0C164E7E7824787A0B310E2A47D5AB6
2016-05-17 21:29:31,046 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:31,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D882215E3BA84A77A204223357F3C09D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:31,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:31,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D882215E3BA84A77A204223357F3C09D
2016-05-17 21:29:31,049 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:31,050 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A0C164E7E7824787A0B310E2A47D5AB6
2016-05-17 21:29:31,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A0C164E7E7824787A0B310E2A47D5AB6
2016-05-17 21:29:31,051 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:31,053 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520570840_0001, with dag id: dag_1463520570840_0001_1
2016-05-17 21:29:31,059 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:31,059 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D882215E3BA84A77A204223357F3C09D
2016-05-17 21:29:31,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D882215E3BA84A77A204223357F3C09D
2016-05-17 21:29:31,061 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CAB32D803EEB44F7A88ABF175C3F0829, ordinal: 0
2016-05-17 21:29:31,062 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAB32D803EEB44F7A88ABF175C3F0829, mem on start (mb), free: 259, total: 360, max: 672
2016-05-17 21:29:31,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:31,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:31,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:31,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:31,063 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:31,063 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:31,063 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:31,064 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:31,064 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:31,064 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:31,064 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A0C164E7E7824787A0B310E2A47D5AB6, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:31,065 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D882215E3BA84A77A204223357F3C09D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:31,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:31,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A0C164E7E7824787A0B310E2A47D5AB6 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:31,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:31,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:31,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A0C164E7E7824787A0B310E2A47D5AB6
2016-05-17 21:29:31,067 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:31,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A0C164E7E7824787A0B310E2A47D5AB6
2016-05-17 21:29:31,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:31,069 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520570840_0001_1_00_000000_0
2016-05-17 21:29:31,069 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:31,070 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1e6b42b4
2016-05-17 21:29:31,071 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1e6b42b4
2016-05-17 21:29:31,071 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:31,071 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:31,072 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:31,072 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: A0C164E7E7824787A0B310E2A47D5AB6
2016-05-17 21:29:31,072 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 56E71CAE731147649A24F908C50039F0
2016-05-17 21:29:31,073 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 56E71CAE731147649A24F908C50039F0
2016-05-17 21:29:31,074 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:31,076 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:31,076 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@4d75cc9; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@570c929a; comparator=cascading.tuple.hadoop.util.TupleComparator@7a6ce91b; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:31,077 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:31,077 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:31,077 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:31,092 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:31,093 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:31,093 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:31,093 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:31,093 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:31,094 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:31,094 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:31,095 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CAB32D803EEB44F7A88ABF175C3F0829, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:31,097 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520570840_0001_1_00_000000_0
2016-05-17 21:29:31,097 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAB32D803EEB44F7A88ABF175C3F0829, mem on close (mb), free: 156, total: 360, max: 672
2016-05-17 21:29:31,098 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:31,098 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:31,098 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:31,098 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:31,101 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:31 Completed running task attempt: attempt_1463520570840_0001_1_00_000000_0
2016-05-17 21:29:31,104 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520570840_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:31,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520570840_0001_1_00, eventCount=1
2016-05-17 21:29:31,105 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520571095, Process_Duration=2, Process_End_Time=1463520571097, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:31,106 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:31,106 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:31,106 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:31,106 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520570840_0001_00_000001
2016-05-17 21:29:31,106 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CAB32D803EEB44F7A88ABF175C3F0829, taskAttemptId=attempt_1463520570840_0001_1_00_000000_0, startTime=1463520571033, finishTime=1463520571105, timeTaken=72, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520571095, Process_Duration=2, Process_End_Time=1463520571097, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:31,107 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520570840_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:31,107 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:31,107 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520570840_0001_1_00_000000_0
2016-05-17 21:29:31,108 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:31,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:TASK_FINISHED]: vertexName=CAB32D803EEB44F7A88ABF175C3F0829, taskId=task_1463520570840_0001_1_00_000000, startTime=1463520571033, finishTime=1463520571108, timeTaken=75, status=SUCCEEDED, successfulAttemptID=attempt_1463520570840_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520571095, Process_Duration=2, Process_End_Time=1463520571097, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:31,108 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520570840_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:31,109 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829] : 1
2016-05-17 21:29:31,109 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829]
2016-05-17 21:29:31,110 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:31,111 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:VERTEX_FINISHED]: vertexName=CAB32D803EEB44F7A88ABF175C3F0829, vertexId=vertex_1463520570840_0001_1_00, initRequestedTime=1463520571007, initedTime=1463520571018, startRequestedTime=1463520571019, startedTime=1463520571019, finishTime=1463520571109, timeTaken=90, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520571095, Process_Duration=2, Process_End_Time=1463520571097, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520571033, firstTasksToStart=[ task_1463520570840_0001_1_00_000000 ], lastTaskFinishTime=1463520571105, lastTasksToFinish=[ task_1463520570840_0001_1_00_000000 ], minTaskDuration=72, maxTaskDuration=72, avgTaskDuration=72.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520570840_0001_1_00_000000 ], longestDurationTasks=[ task_1463520570840_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:31,111 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:31,111 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570840_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:31,111 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:31,111 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520570840_0001_00_000001, stoppedTime=1463520571111, exitStatus=0
2016-05-17 21:29:31,112 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570840_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:31,112 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D] attempt: attempt_1463520570840_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:31,112 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D882215E3BA84A77A204223357F3C09D
2016-05-17 21:29:31,112 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D]
2016-05-17 21:29:31,112 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:31,112 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520570840_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:31,113 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:31,113 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:TASK_STARTED]: vertexName=D882215E3BA84A77A204223357F3C09D, taskId=task_1463520570840_0001_1_01_000000, scheduledTime=1463520571113, launchTime=1463520571113
2016-05-17 21:29:31,113 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520570840_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:31,113 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D]
2016-05-17 21:29:31,113 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570840_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:31,113 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7FB087F25B6A415483E3D9709C0CBB5D/AB1CB9EE09C04EEE98507937BA1B33AA] every/(1/1) ...latform/-1_VALUES_RESULTS, VertexName: D882215E3BA84A77A204223357F3C09D, VertexParallelism: 1, TaskAttemptID:attempt_1463520570840_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CAB32D803EEB44F7A88ABF175C3F0829, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=313EB353AC88421999DB4BAED6340038, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:31,114 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520570840_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:31,114 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:31,114 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520570840_0001_00_000002
2016-05-17 21:29:31,114 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570840_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:31,116 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520570840_0001_00_000002
2016-05-17 21:29:31,116 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:31,116 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570840_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:31,117 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:31,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520570840_0001_00_000002, launchTime=1463520571116
2016-05-17 21:29:31,117 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520570840_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:31,118 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520570840_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:31,117 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520570840_0001_00_000002
2016-05-17 21:29:31,118 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520570840_0001_00_000001, Exiting
2016-05-17 21:29:31,118 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520570840_0001_00_000001
2016-05-17 21:29:31,118 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520570840_0001_00_000001
2016-05-17 21:29:31,118 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520570840_0001_1_01_000000_0] to container: [container_1463520570840_0001_00_000002]
2016-05-17 21:29:31,119 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570840_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:31,121 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520570840_0001_1_01_000000_0] started. Is using containerId: [container_1463520570840_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:31,121 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520570840_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:31,122 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520570840_0001_00_000001 completed successfully
2016-05-17 21:29:31,121 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520570840_0001_00_000002 given task: attempt_1463520570840_0001_1_01_000000_0
2016-05-17 21:29:31,122 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520570840_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520570840_0001_1_01_000000_0
2016-05-17 21:29:31,121 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:31 Starting to run new task attempt: attempt_1463520570840_0001_1_01_000000_0
2016-05-17 21:29:31,121 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D882215E3BA84A77A204223357F3C09D, taskAttemptId=attempt_1463520570840_0001_1_01_000000_0, startTime=1463520571121, containerId=container_1463520570840_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520570840_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:31,124 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520570840_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:31,124 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570840_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:31,124 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520570840_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:31,124 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520570840_0001_1_01_000000_0
2016-05-17 21:29:31,124 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:31,125 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:31,125 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:31,125 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:31,125 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:31,126 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:31,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7FB087F25B6A415483E3D9709C0CBB5D/AB1CB9EE09C04EEE98507937BA1B33AA] every/(1/1) ...latform/-1_VALUES_RESULTS, VertexName: D882215E3BA84A77A204223357F3C09D, VertexParallelism: 1, TaskAttemptID:attempt_1463520570840_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CAB32D803EEB44F7A88ABF175C3F0829, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=313EB353AC88421999DB4BAED6340038, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:31,126 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:31,127 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520570840_0001_1_01_000000_0
2016-05-17 21:29:31,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:31,127 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520570840_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:31,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:31,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CAB32D803EEB44F7A88ABF175C3F0829, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:31,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=313EB353AC88421999DB4BAED6340038, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:31,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:31,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CAB32D803EEB44F7A88ABF175C3F0829
2016-05-17 21:29:31,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:31,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 313EB353AC88421999DB4BAED6340038
2016-05-17 21:29:31,129 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:31,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:31,130 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 313EB353AC88421999DB4BAED6340038
2016-05-17 21:29:31,133 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:31,133 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:31,133 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CAB32D803EEB44F7A88ABF175C3F0829
2016-05-17 21:29:31,134 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:31,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CAB32D803EEB44F7A88ABF175C3F0829
2016-05-17 21:29:31,141 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:31,141 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 313EB353AC88421999DB4BAED6340038
2016-05-17 21:29:31,142 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D882215E3BA84A77A204223357F3C09D, ordinal: 1
2016-05-17 21:29:31,142 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D882215E3BA84A77A204223357F3C09D, mem on start (mb), free: 150, total: 360, max: 672
2016-05-17 21:29:31,143 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:31,143 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:31,143 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:31,143 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:31,143 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:31,143 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:31,143 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:31,143 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:31,143 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:31,144 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:31,144 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 313EB353AC88421999DB4BAED6340038, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:31,144 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CAB32D803EEB44F7A88ABF175C3F0829, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:31,144 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:31,144 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CAB32D803EEB44F7A88ABF175C3F0829 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:31,144 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:31,144 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:31,144 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CAB32D803EEB44F7A88ABF175C3F0829
2016-05-17 21:29:31,146 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:31,146 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:31,147 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CAB32D803EEB44F7A88ABF175C3F0829 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:31,148 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:31,148 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:31,149 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:31,149 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CAB32D803EEB44F7A88ABF175C3F0829
2016-05-17 21:29:31,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:31,150 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520570840_0001_1_01_000000_0
2016-05-17 21:29:31,150 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520570840_0001_1_00_000000_0_10069, runDuration: 53468, ]
2016-05-17 21:29:31,152 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520570840_0001_1_00_000000_0_10069] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520571152, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:31,152 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CAB32D803EEB44F7A88ABF175C3F0829
2016-05-17 21:29:31,152 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:31,152 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [CAB32D803EEB44F7A88ABF175C3F0829] #12 in 1ms
2016-05-17 21:29:31,153 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:31,153 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CAB32D803EEB44F7A88ABF175C3F0829] #12, status:false, isInterrupted:false
2016-05-17 21:29:31,154 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:31,154 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 56E71CAE731147649A24F908C50039F0
2016-05-17 21:29:31,154 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_RESULTS"], id: 313EB353AC88421999DB4BAED6340038
2016-05-17 21:29:31,154 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_RESULTS"]
2016-05-17 21:29:31,155 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 56E71CAE731147649A24F908C50039F0, for 1 inputs
2016-05-17 21:29:31,156 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:31,156 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:31,157 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:31,157 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:31,158 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:31,158 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D882215E3BA84A77A204223357F3C09D, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:31,158 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CAB32D803EEB44F7A88ABF175C3F0829
2016-05-17 21:29:31,158 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:31,160 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:31,162 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@dd2e4e; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@390b0353
2016-05-17 21:29:31,163 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:31,165 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:31,169 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:31,169 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:31,170 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520570840_0001_1_01_000000_0
2016-05-17 21:29:31,170 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520570840_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:31,170 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520570840_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:31,176 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:31,176 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D882215E3BA84A77A204223357F3C09D, taskAttemptId=attempt_1463520570840_0001_1_01_000000_0, startTime=1463520571121, finishTime=1463520571176, timeTaken=55, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520571158, Process_Duration=12, Process_End_Time=1463520571170, Read_Duration=1, Tuples_Read=1
2016-05-17 21:29:31,176 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520570840_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:31,177 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520570840_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:31,177 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:TASK_FINISHED]: vertexName=D882215E3BA84A77A204223357F3C09D, taskId=task_1463520570840_0001_1_01_000000, startTime=1463520571121, finishTime=1463520571177, timeTaken=56, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:31,177 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520570840_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:31,177 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D] : 1
2016-05-17 21:29:31,177 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D] because task failed: task_1463520570840_0001_1_01_000000
2016-05-17 21:29:31,178 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:31,178 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:31,178 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D]
2016-05-17 21:29:31,178 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=313EB353AC88421999DB4BAED6340038, vertexId=vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D]
2016-05-17 21:29:31,179 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:31,180 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:VERTEX_FINISHED]: vertexName=D882215E3BA84A77A204223357F3C09D, vertexId=vertex_1463520570840_0001_1_01, initRequestedTime=1463520571008, initedTime=1463520571015, startRequestedTime=1463520571021, startedTime=1463520571021, finishTime=1463520571179, timeTaken=158, status=FAILED, diagnostics=Task failed, taskId=task_1463520570840_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520571121, firstTasksToStart=[ task_1463520570840_0001_1_01_000000 ], lastTaskFinishTime=1463520571176, lastTasksToFinish=[ task_1463520570840_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:31,180 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:31,181 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520570840_0001_00_000002, stoppedTime=1463520571181, exitStatus=0
2016-05-17 21:29:31,181 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570840_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:31,181 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520570840_0001_1_01_000000_0 failed: true container: container_1463520570840_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:31,181 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:31,181 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:31,181 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520570840_0001_1
2016-05-17 21:29:31,181 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 313EB353AC88421999DB4BAED6340038 for vertex: vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D]
2016-05-17 21:29:31,182 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520570840_0001_1_00 [CAB32D803EEB44F7A88ABF175C3F0829]
2016-05-17 21:29:31,182 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:31,182 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:31,183 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:31,183 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520570840_0001_1, queueSize=0
2016-05-17 21:29:31,184 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:31,185 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520570840_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520570840_0001_1, startTime=1463520571006, finishTime=1463520571182, timeTaken=176, status=FAILED, diagnostics=Vertex failed, vertexName=D882215E3BA84A77A204223357F3C09D, vertexId=vertex_1463520570840_0001_1_01, diagnostics=[Task failed, taskId=task_1463520570840_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520570840_0001_1_01 [D882215E3BA84A77A204223357F3C09D] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520571095, Process_Duration=2, Process_End_Time=1463520571097, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:31,185 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520570840_0001_1 finished with state: FAILED
2016-05-17 21:29:31,185 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520570840_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:31,185 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520571158, Process_Duration=12, Process_End_Time=1463520571170, Read_Duration=1, Tuples_Read=1]]
2016-05-17 21:29:31,186 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D882215E3BA84A77A204223357F3C09D, mem on close (mb), free: 143, total: 360, max: 672
2016-05-17 21:29:31,186 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=D882215E3BA84A77A204223357F3C09D, index=1
2016-05-17 21:29:31 Completed Dag: dag_1463520570840_0001_1
2016-05-17 21:29:31,187 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520570840_0001_1, dagState=FAILED
2016-05-17 21:29:31,187 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520570840_0001_1_post
2016-05-17 21:29:31,187 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:31,187 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:31,187 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520570840_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:31,188 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CAB32D803EEB44F7A88ABF175C3F0829
2016-05-17 21:29:31,188 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:31,188 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570840_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:31,191 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:31,191 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:31,191 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:31,191 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:31,192 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:31,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=D882215E3BA84A77A204223357F3C09D, sourceVertex=CAB32D803EEB44F7A88ABF175C3F0829
2016-05-17 21:29:31,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=D882215E3BA84A77A204223357F3C09D, sourceVertex=313EB353AC88421999DB4BAED6340038
2016-05-17 21:29:31,193 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520570840_0001_00_000002
2016-05-17 21:29:31,194 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520570840_0001_00_000002
2016-05-17 21:29:31,194 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520570840_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:31,194 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520570840_0001_00_000002 completed but with errors
2016-05-17 21:29:31,195 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520570840_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:31,195 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520570840_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:31,195 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520570840_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:31,195 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/-1_VALUES_RESULTS, applicationId=application_1463520570840_0001
2016-05-17 21:29:31,197 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:31,197 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:31,197 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:31,197 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:31,197 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:31,198 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:31,198 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:31,199 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:31,199 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:31,199 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:31,199 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:31,199 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:31,200 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:31,205 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7FB087F25B6A415483E3D9709C0CBB5D/AB1CB9EE09C04EEE98507937BA1B33AA/.tez/application_1463520570840_0001
2016-05-17 21:29:31,206 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:31,206 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...latform/-1_VALUES_RESULTS
2016-05-17 21:29:31,207 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:31,207 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_RESULTS/_temporary
2016-05-17 21:29:31,208 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.385
2016-05-17 21:29:31,208 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_RESULTS
2016-05-17 21:29:31,237 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:31,237 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:31,276 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 21:29:31,287 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-17 21:29:31,287 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:31,288 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:31,288 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:31,314 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:31,319 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_ALL
2016-05-17 21:29:31,367 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:31,368 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:31,368 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_ALL"]
2016-05-17 21:29:31,368 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:31,368 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:31,368 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:31,369 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...blyplatform/-1_VALUES_ALL
2016-05-17 21:29:31,369 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:31,381 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:31,386 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:31,386 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:31,386 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:31,395 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6EFD18ADFC9945E6A363DDB8370EC793/C6C6CDF8B36C43F49207E25C0578AE52/.tez/application_1463520571386_0001 doesn't exist and is created
2016-05-17 21:29:31,409 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:31,410 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:31,411 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6EFD18ADFC9945E6A363DDB8370EC793/C6C6CDF8B36C43F49207E25C0578AE52/.tez/application_1463520571386_0001
2016-05-17 21:29:31,414 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520571386_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:31,423 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:31,424 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:31,426 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:31,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:31,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6EFD18ADFC9945E6A363DDB8370EC793/C6C6CDF8B36C43F49207E25C0578AE52/.tez/application_1463520571386_0001/localmode-log-dir
2016-05-17 21:29:31,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6EFD18ADFC9945E6A363DDB8370EC793/C6C6CDF8B36C43F49207E25C0578AE52/.tez/application_1463520571386_0001/localmode-log-dir/history.txt.appattempt_1463520571386_0001_000000, maxErrors=10
2016-05-17 21:29:31,426 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:31,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520571386_0001_000000, appSubmitTime=1463520571413, launchTime=1463520571413
2016-05-17 21:29:31,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:31,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:31,436 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54218
2016-05-17 21:29:31,437 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:31,437 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:31,439 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:31,441 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520571386_0001_000000, startTime=1463520571441
2016-05-17 21:29:31,441 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:31,441 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:31,469 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:31,469 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:31,510 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:31,510 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:31,510 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/-1_VALUES_ALL, applicationId=application_1463520571386_0001, dagName=[6EFD18ADFC9945E6A363DDB8370EC793/C6C6CDF8B36C43F49207E25C0578AE52] every/(1/1) ...blyplatform/-1_VALUES_ALL
2016-05-17 21:29:31,525 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [6EFD18ADFC9945E6A363DDB8370EC793/C6C6CDF8B36C43F49207E25C0578AE52] every/(1/1) ...blyplatform/-1_VALUES_ALL
2016-05-17 21:29:31,526 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520571386_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6EFD18ADFC9945E6A363DDB8370EC793/C6C6CDF8B36C43F49207E25C0578AE52/.tez/application_1463520571386_0001/localmode-log-dir/dag_1463520571386_0001_1.dot
2016-05-17 21:29:31,527 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6EFD18ADFC9945E6A363DDB8370EC793/C6C6CDF8B36C43F49207E25C0578AE52/.tez/application_1463520571386_0001/localmode-log-dir/dag_1463520571386_0001_1-tez-dag.pb.txt
2016-05-17 21:29:31,534 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520571386_0001_1
2016-05-17 21:29:31,534 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:31,534 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [6EFD18ADFC9945E6A363DDB8370EC793/C6C6CDF8B36C43F49207E25C0578AE52] every/(1/1) ...blyplatform/-1_VALUES_ALL
2016-05-17 21:29:31 Running Dag: dag_1463520571386_0001_1
2016-05-17 21:29:31,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520571386_0001_1, submitTime=1463520571525
2016-05-17 21:29:31,545 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:31,546 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FF7EAECBC68A4D1B9BBC9FA928AB72B4
2016-05-17 21:29:31,546 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:31,546 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520571386_0001_1, initTime=1463520571545
2016-05-17 21:29:31,546 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:31,547 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520571386_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:31,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:DAG_STARTED]: dagID=dag_1463520571386_0001_1, startTime=1463520571547
2016-05-17 21:29:31,548 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:31,547 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:31,548 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520571386_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:31,549 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BDCA7D52684E4383ACAFF6A2810914F6 : {13E90D55829F4D4994B89B37B124C45D={InputName=13E90D55829F4D4994B89B37B124C45D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:31,549 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 13E90D55829F4D4994B89B37B124C45D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:31,549 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6]
2016-05-17 21:29:31,549 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6]
2016-05-17 21:29:31,549 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/-1_VALUES_ALL, applicationId=application_1463520571386_0001, dagName=[6EFD18ADFC9945E6A363DDB8370EC793/C6C6CDF8B36C43F49207E25C0578AE52] every/(1/1) ...blyplatform/-1_VALUES_ALL
2016-05-17 21:29:31,549 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6]
2016-05-17 21:29:31,550 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6]. Starting root input initializers: 1
2016-05-17 21:29:31,551 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:31,551 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4]
2016-05-17 21:29:31,552 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 13E90D55829F4D4994B89B37B124C45D on vertex vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6]
2016-05-17 21:29:31,554 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:31,554 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:31,554 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4]
2016-05-17 21:29:31,555 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4]
2016-05-17 21:29:31,555 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4]
2016-05-17 21:29:31,555 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9DB1B8CE93304E58946B73DE573BC0F0, vertexId=vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:31,555 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9DB1B8CE93304E58946B73DE573BC0F0, vertexId=vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4]
2016-05-17 21:29:31,554 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 13E90D55829F4D4994B89B37B124C45D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:31,557 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:31,557 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:31,557 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9DB1B8CE93304E58946B73DE573BC0F0, vertexName=FF7EAECBC68A4D1B9BBC9FA928AB72B4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:31,558 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9DB1B8CE93304E58946B73DE573BC0F0, vertexId=vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4]
2016-05-17 21:29:31,559 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:31,559 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 13E90D55829F4D4994B89B37B124C45D on vertex vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6]
2016-05-17 21:29:31,559 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:31,560 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FF7EAECBC68A4D1B9BBC9FA928AB72B4, vertexId=vertex_1463520571386_0001_1_01, initRequestedTime=1463520571551, initedTime=1463520571559, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:31,560 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:31,561 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:31,561 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {13E90D55829F4D4994B89B37B124C45D=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:31,561 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6] parallelism set to 1
2016-05-17 21:29:31,562 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:31,563 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520571386_0001_1_00, eventCount=1
2016-05-17 21:29:31,563 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:31,564 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BDCA7D52684E4383ACAFF6A2810914F6, vertexId=vertex_1463520571386_0001_1_00, initRequestedTime=1463520571548, initedTime=1463520571563, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:31,564 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6] with distanceFromRoot: 0
2016-05-17 21:29:31,565 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:31,565 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BDCA7D52684E4383ACAFF6A2810914F6
2016-05-17 21:29:31,565 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6]
2016-05-17 21:29:31,566 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:31,570 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520571386_0001_1_00, startRequestedTime=1463520571565, startedTime=1463520571565
2016-05-17 21:29:31,571 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:31,571 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:TASK_STARTED]: vertexName=BDCA7D52684E4383ACAFF6A2810914F6, taskId=task_1463520571386_0001_1_00_000000, scheduledTime=1463520571571, launchTime=1463520571571
2016-05-17 21:29:31,572 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520571386_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:31,573 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520571386_0001_1_00 for vertex: vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4] numStartedSources: 1 numSources: 1
2016-05-17 21:29:31,574 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4] with distanceFromRoot: 1
2016-05-17 21:29:31,572 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:31,572 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520571386_0001, with dag id: dag_1463520571386_0001_1
2016-05-17 21:29:31,574 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6]
2016-05-17 21:29:31,576 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BDCA7D52684E4383ACAFF6A2810914F6 in vertex: FF7EAECBC68A4D1B9BBC9FA928AB72B4
2016-05-17 21:29:31,576 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FF7EAECBC68A4D1B9BBC9FA928AB72B4 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:31,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:31,577 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520571386_0001_1_01, startRequestedTime=1463520571576, startedTime=1463520571576
2016-05-17 21:29:31,577 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:31,579 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6EFD18ADFC9945E6A363DDB8370EC793/C6C6CDF8B36C43F49207E25C0578AE52] every/(1/1) ...blyplatform/-1_VALUES_ALL, VertexName: BDCA7D52684E4383ACAFF6A2810914F6, VertexParallelism: 1, TaskAttemptID:attempt_1463520571386_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=13E90D55829F4D4994B89B37B124C45D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FF7EAECBC68A4D1B9BBC9FA928AB72B4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:31,581 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520571386_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:31,582 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:31,582 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:31,583 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:31,583 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:31,584 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520571386_0001_00_000001
2016-05-17 21:29:31,584 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571386_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:31,586 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520571386_0001_00_000001
2016-05-17 21:29:31,586 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:31,588 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571386_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:31,589 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:31,589 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520571386_0001_00_000001, launchTime=1463520571588
2016-05-17 21:29:31,590 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520571386_0001_00_000001
2016-05-17 21:29:31,591 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520571386_0001_1_00_000000_0] to container: [container_1463520571386_0001_00_000001]
2016-05-17 21:29:31,591 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571386_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:31,591 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520571386_0001_00_000001 given task: attempt_1463520571386_0001_1_00_000000_0
2016-05-17 21:29:31,592 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520571386_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520571386_0001_1_00_000000_0
2016-05-17 21:29:31,592 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520571386_0001_1_00_000000_0] started. Is using containerId: [container_1463520571386_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:31,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BDCA7D52684E4383ACAFF6A2810914F6, taskAttemptId=attempt_1463520571386_0001_1_00_000000_0, startTime=1463520571592, containerId=container_1463520571386_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520571386_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:31,592 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520571386_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:31,592 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520571386_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:31 Starting to run new task attempt: attempt_1463520571386_0001_1_00_000000_0
2016-05-17 21:29:31,593 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:31,593 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520571386_0001_1_00_000000_0
2016-05-17 21:29:31,593 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:31,593 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:31,593 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:31,594 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:31,594 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:31,594 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:31,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6EFD18ADFC9945E6A363DDB8370EC793/C6C6CDF8B36C43F49207E25C0578AE52] every/(1/1) ...blyplatform/-1_VALUES_ALL, VertexName: BDCA7D52684E4383ACAFF6A2810914F6, VertexParallelism: 1, TaskAttemptID:attempt_1463520571386_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=13E90D55829F4D4994B89B37B124C45D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FF7EAECBC68A4D1B9BBC9FA928AB72B4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:31,595 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:31,595 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520571386_0001_1_00_000000_0
2016-05-17 21:29:31,595 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:31,595 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:31,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=13E90D55829F4D4994B89B37B124C45D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:31,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:31,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 13E90D55829F4D4994B89B37B124C45D
2016-05-17 21:29:31,597 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 13E90D55829F4D4994B89B37B124C45D
2016-05-17 21:29:31,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:31,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FF7EAECBC68A4D1B9BBC9FA928AB72B4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:31,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:31,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FF7EAECBC68A4D1B9BBC9FA928AB72B4
2016-05-17 21:29:31,600 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:31,600 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FF7EAECBC68A4D1B9BBC9FA928AB72B4
2016-05-17 21:29:31,600 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:31,604 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520571386_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:31,605 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FF7EAECBC68A4D1B9BBC9FA928AB72B4
2016-05-17 21:29:31,607 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:31,607 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 13E90D55829F4D4994B89B37B124C45D
2016-05-17 21:29:31,608 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 13E90D55829F4D4994B89B37B124C45D
2016-05-17 21:29:31,609 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BDCA7D52684E4383ACAFF6A2810914F6, ordinal: 0
2016-05-17 21:29:31,610 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDCA7D52684E4383ACAFF6A2810914F6, mem on start (mb), free: 268, total: 360, max: 672
2016-05-17 21:29:31,610 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:31,610 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:31,610 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:31,610 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:31,610 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:31,611 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:31,611 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:31,611 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:31,611 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:31,611 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:31,611 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 13E90D55829F4D4994B89B37B124C45D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:31,612 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FF7EAECBC68A4D1B9BBC9FA928AB72B4, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:31,612 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:31,612 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 13E90D55829F4D4994B89B37B124C45D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:31,612 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:31,612 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:31,612 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 13E90D55829F4D4994B89B37B124C45D
2016-05-17 21:29:31,613 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:31,613 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 13E90D55829F4D4994B89B37B124C45D
2016-05-17 21:29:31,613 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:31,613 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520571386_0001_1_00_000000_0
2016-05-17 21:29:31,614 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:31,614 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@cb1b79
2016-05-17 21:29:31,615 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@cb1b79
2016-05-17 21:29:31,615 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:31,616 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:31,616 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:31,617 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 13E90D55829F4D4994B89B37B124C45D
2016-05-17 21:29:31,617 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 74DC198450FB499FB4DA1BB8EE5EA9DE
2016-05-17 21:29:31,617 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 74DC198450FB499FB4DA1BB8EE5EA9DE
2016-05-17 21:29:31,619 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:31,622 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:31,622 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@169c0c37; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@5ee5fa79; comparator=cascading.tuple.hadoop.util.TupleComparator@6ee1c969; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:31,622 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:31,622 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:31,623 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:31,635 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:31,636 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:31,636 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:31,636 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:31,636 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:31,638 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:31,638 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:31,639 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BDCA7D52684E4383ACAFF6A2810914F6, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:31,641 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520571386_0001_1_00_000000_0
2016-05-17 21:29:31,641 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDCA7D52684E4383ACAFF6A2810914F6, mem on close (mb), free: 165, total: 360, max: 672
2016-05-17 21:29:31,641 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:31,642 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:31,642 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:31,642 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:31,646 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:31 Completed running task attempt: attempt_1463520571386_0001_1_00_000000_0
2016-05-17 21:29:31,649 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520571386_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:31,652 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520571639, Process_Duration=2, Process_End_Time=1463520571641, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:31,652 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520571386_0001_1_00, eventCount=1
2016-05-17 21:29:31,652 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:31,653 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:31,653 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520571386_0001_00_000001
2016-05-17 21:29:31,653 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BDCA7D52684E4383ACAFF6A2810914F6, taskAttemptId=attempt_1463520571386_0001_1_00_000000_0, startTime=1463520571592, finishTime=1463520571652, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520571639, Process_Duration=2, Process_End_Time=1463520571641, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:31,653 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:31,654 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520571386_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:31,654 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:31,654 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520571386_0001_1_00_000000_0
2016-05-17 21:29:31,655 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:31,655 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:TASK_FINISHED]: vertexName=BDCA7D52684E4383ACAFF6A2810914F6, taskId=task_1463520571386_0001_1_00_000000, startTime=1463520571592, finishTime=1463520571655, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1463520571386_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520571639, Process_Duration=2, Process_End_Time=1463520571641, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:31,656 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520571386_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:31,656 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6] : 1
2016-05-17 21:29:31,656 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6]
2016-05-17 21:29:31,657 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:31,657 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:VERTEX_FINISHED]: vertexName=BDCA7D52684E4383ACAFF6A2810914F6, vertexId=vertex_1463520571386_0001_1_00, initRequestedTime=1463520571548, initedTime=1463520571563, startRequestedTime=1463520571565, startedTime=1463520571565, finishTime=1463520571656, timeTaken=91, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520571639, Process_Duration=2, Process_End_Time=1463520571641, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520571592, firstTasksToStart=[ task_1463520571386_0001_1_00_000000 ], lastTaskFinishTime=1463520571652, lastTasksToFinish=[ task_1463520571386_0001_1_00_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520571386_0001_1_00_000000 ], longestDurationTasks=[ task_1463520571386_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:31,657 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:31,658 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571386_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:31,658 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520571386_0001_00_000001, stoppedTime=1463520571658, exitStatus=0
2016-05-17 21:29:31,658 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:31,658 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571386_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:31,658 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4] attempt: attempt_1463520571386_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:31,658 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FF7EAECBC68A4D1B9BBC9FA928AB72B4
2016-05-17 21:29:31,659 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4]
2016-05-17 21:29:31,659 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:31,659 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520571386_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:31,659 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:TASK_STARTED]: vertexName=FF7EAECBC68A4D1B9BBC9FA928AB72B4, taskId=task_1463520571386_0001_1_01_000000, scheduledTime=1463520571659, launchTime=1463520571659
2016-05-17 21:29:31,659 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:31,660 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520571386_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:31,661 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571386_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:31,661 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4]
2016-05-17 21:29:31,662 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6EFD18ADFC9945E6A363DDB8370EC793/C6C6CDF8B36C43F49207E25C0578AE52] every/(1/1) ...blyplatform/-1_VALUES_ALL, VertexName: FF7EAECBC68A4D1B9BBC9FA928AB72B4, VertexParallelism: 1, TaskAttemptID:attempt_1463520571386_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BDCA7D52684E4383ACAFF6A2810914F6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9DB1B8CE93304E58946B73DE573BC0F0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:31,662 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520571386_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:31,662 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:31,663 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520571386_0001_00_000002
2016-05-17 21:29:31,663 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571386_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:31,664 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520571386_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:31,664 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520571386_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:31,665 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520571386_0001_00_000001, Exiting
2016-05-17 21:29:31,665 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520571386_0001_00_000001
2016-05-17 21:29:31,665 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520571386_0001_00_000001
2016-05-17 21:29:31,665 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520571386_0001_00_000002
2016-05-17 21:29:31,666 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:31,667 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520571386_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:31,667 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520571386_0001_00_000001 completed successfully
2016-05-17 21:29:31,667 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571386_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:31,668 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571386_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:31,669 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:31,669 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520571386_0001_00_000002, launchTime=1463520571668
2016-05-17 21:29:31,670 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520571386_0001_00_000002
2016-05-17 21:29:31,670 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520571386_0001_1_01_000000_0] to container: [container_1463520571386_0001_00_000002]
2016-05-17 21:29:31,670 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571386_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:31,671 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520571386_0001_1_01_000000_0] started. Is using containerId: [container_1463520571386_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:31,671 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FF7EAECBC68A4D1B9BBC9FA928AB72B4, taskAttemptId=attempt_1463520571386_0001_1_01_000000_0, startTime=1463520571670, containerId=container_1463520571386_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520571386_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:31,671 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520571386_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:31,671 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520571386_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:31,671 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:31,672 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520571386_0001_00_000002 given task: attempt_1463520571386_0001_1_01_000000_0
2016-05-17 21:29:31,672 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520571386_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520571386_0001_1_01_000000_0
2016-05-17 21:29:31 Starting to run new task attempt: attempt_1463520571386_0001_1_01_000000_0
2016-05-17 21:29:31,672 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520571386_0001_1_01_000000_0
2016-05-17 21:29:31,673 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:31,673 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:31,673 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:31,674 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:31,674 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:31,674 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:31,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6EFD18ADFC9945E6A363DDB8370EC793/C6C6CDF8B36C43F49207E25C0578AE52] every/(1/1) ...blyplatform/-1_VALUES_ALL, VertexName: FF7EAECBC68A4D1B9BBC9FA928AB72B4, VertexParallelism: 1, TaskAttemptID:attempt_1463520571386_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BDCA7D52684E4383ACAFF6A2810914F6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9DB1B8CE93304E58946B73DE573BC0F0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:31,675 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:31,675 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520571386_0001_1_01_000000_0
2016-05-17 21:29:31,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:31,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:31,676 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520571386_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:31,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BDCA7D52684E4383ACAFF6A2810914F6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:31,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:31,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BDCA7D52684E4383ACAFF6A2810914F6
2016-05-17 21:29:31,678 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:31,678 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:31,678 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BDCA7D52684E4383ACAFF6A2810914F6
2016-05-17 21:29:31,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BDCA7D52684E4383ACAFF6A2810914F6
2016-05-17 21:29:31,682 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:31,684 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9DB1B8CE93304E58946B73DE573BC0F0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:31,684 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:31,684 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:31,685 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9DB1B8CE93304E58946B73DE573BC0F0
2016-05-17 21:29:31,685 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:31,686 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9DB1B8CE93304E58946B73DE573BC0F0
2016-05-17 21:29:31,695 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FF7EAECBC68A4D1B9BBC9FA928AB72B4, ordinal: 1
2016-05-17 21:29:31,695 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF7EAECBC68A4D1B9BBC9FA928AB72B4, mem on start (mb), free: 158, total: 360, max: 672
2016-05-17 21:29:31,695 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:31,695 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:31,695 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:31,697 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:31,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9DB1B8CE93304E58946B73DE573BC0F0
2016-05-17 21:29:31,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:31,697 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:31,697 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:31,697 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:31,697 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:31,698 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:31,698 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:31,698 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BDCA7D52684E4383ACAFF6A2810914F6, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:31,698 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9DB1B8CE93304E58946B73DE573BC0F0, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:31,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:31,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BDCA7D52684E4383ACAFF6A2810914F6
2016-05-17 21:29:31,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BDCA7D52684E4383ACAFF6A2810914F6 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:31,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:31,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:31,699 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:31,700 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:31,700 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BDCA7D52684E4383ACAFF6A2810914F6 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:31,700 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:31,700 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:31,701 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:31,702 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BDCA7D52684E4383ACAFF6A2810914F6
2016-05-17 21:29:31,702 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:31,702 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520571386_0001_1_01_000000_0
2016-05-17 21:29:31,704 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520571386_0001_1_00_000000_0_10075, runDuration: 49234, ]
2016-05-17 21:29:31,705 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520571386_0001_1_00_000000_0_10075] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520571705, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:31,706 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:31,706 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BDCA7D52684E4383ACAFF6A2810914F6
2016-05-17 21:29:31,706 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:31,706 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 74DC198450FB499FB4DA1BB8EE5EA9DE
2016-05-17 21:29:31,706 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [BDCA7D52684E4383ACAFF6A2810914F6] #13 in 1ms
2016-05-17 21:29:31,707 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_ALL"], id: 9DB1B8CE93304E58946B73DE573BC0F0
2016-05-17 21:29:31,707 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_ALL"]
2016-05-17 21:29:31,707 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 74DC198450FB499FB4DA1BB8EE5EA9DE, for 1 inputs
2016-05-17 21:29:31,707 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BDCA7D52684E4383ACAFF6A2810914F6] #13, status:false, isInterrupted:false
2016-05-17 21:29:31,708 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:31,709 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:31,709 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:31,709 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:31,710 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:31,710 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:31,710 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BDCA7D52684E4383ACAFF6A2810914F6
2016-05-17 21:29:31,710 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:31,710 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FF7EAECBC68A4D1B9BBC9FA928AB72B4, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:31,712 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:31,714 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@5710247b; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@64cb33e4
2016-05-17 21:29:31,715 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:31,716 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:31,717 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:31,717 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:31,717 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520571386_0001_1_01_000000_0
2016-05-17 21:29:31,717 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520571386_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:31,718 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520571386_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:31,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520571710, Process_Duration=8, Process_End_Time=1463520571718, Read_Duration=0, Tuples_Read=1]]
2016-05-17 21:29:31,722 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:31,722 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF7EAECBC68A4D1B9BBC9FA928AB72B4, mem on close (mb), free: 152, total: 360, max: 672
2016-05-17 21:29:31,722 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FF7EAECBC68A4D1B9BBC9FA928AB72B4, taskAttemptId=attempt_1463520571386_0001_1_01_000000_0, startTime=1463520571670, finishTime=1463520571722, timeTaken=52, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520571710, Process_Duration=8, Process_End_Time=1463520571718, Read_Duration=0, Tuples_Read=1
2016-05-17 21:29:31,723 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520571386_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:31,723 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520571386_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:31,723 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:TASK_FINISHED]: vertexName=FF7EAECBC68A4D1B9BBC9FA928AB72B4, taskId=task_1463520571386_0001_1_01_000000, startTime=1463520571670, finishTime=1463520571723, timeTaken=53, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:31,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=FF7EAECBC68A4D1B9BBC9FA928AB72B4, index=1
2016-05-17 21:29:31,723 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520571386_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:31,724 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4] : 1
2016-05-17 21:29:31,724 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:31,723 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:31,724 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4] because task failed: task_1463520571386_0001_1_01_000000
2016-05-17 21:29:31,724 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:31,724 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:31,725 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4]
2016-05-17 21:29:31,725 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=9DB1B8CE93304E58946B73DE573BC0F0, vertexId=vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4]
2016-05-17 21:29:31,726 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:31,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:VERTEX_FINISHED]: vertexName=FF7EAECBC68A4D1B9BBC9FA928AB72B4, vertexId=vertex_1463520571386_0001_1_01, initRequestedTime=1463520571551, initedTime=1463520571559, startRequestedTime=1463520571576, startedTime=1463520571576, finishTime=1463520571726, timeTaken=150, status=FAILED, diagnostics=Task failed, taskId=task_1463520571386_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520571670, firstTasksToStart=[ task_1463520571386_0001_1_01_000000 ], lastTaskFinishTime=1463520571722, lastTasksToFinish=[ task_1463520571386_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:31,727 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:31,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520571386_0001_00_000002, stoppedTime=1463520571727, exitStatus=0
2016-05-17 21:29:31,727 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571386_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:31,727 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:31,728 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520571386_0001_1_01_000000_0 failed: true container: container_1463520571386_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:31,728 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:31,728 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:31,728 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520571386_0001_1
2016-05-17 21:29:31,728 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520571386_0001_1_00 [BDCA7D52684E4383ACAFF6A2810914F6]
2016-05-17 21:29:31,728 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 9DB1B8CE93304E58946B73DE573BC0F0 for vertex: vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4]
2016-05-17 21:29:31,729 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BDCA7D52684E4383ACAFF6A2810914F6
2016-05-17 21:29:31,730 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:31,731 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520571386_0001_1, queueSize=0
2016-05-17 21:29:31,731 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:31,732 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571386_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520571386_0001_1, startTime=1463520571547, finishTime=1463520571729, timeTaken=182, status=FAILED, diagnostics=Vertex failed, vertexName=FF7EAECBC68A4D1B9BBC9FA928AB72B4, vertexId=vertex_1463520571386_0001_1_01, diagnostics=[Task failed, taskId=task_1463520571386_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520571386_0001_1_01 [FF7EAECBC68A4D1B9BBC9FA928AB72B4] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520571639, Process_Duration=2, Process_End_Time=1463520571641, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:31,732 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520571386_0001_1 finished with state: FAILED
2016-05-17 21:29:31,732 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520571386_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:31,732 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520571386_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:31,733 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:31,733 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:31,733 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:31,733 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:31,734 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:31,735 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=FF7EAECBC68A4D1B9BBC9FA928AB72B4, sourceVertex=BDCA7D52684E4383ACAFF6A2810914F6
2016-05-17 21:29:31,735 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=FF7EAECBC68A4D1B9BBC9FA928AB72B4, sourceVertex=9DB1B8CE93304E58946B73DE573BC0F0
2016-05-17 21:29:31,735 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520571386_0001_00_000002
2016-05-17 21:29:31,735 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520571386_0001_00_000002
2016-05-17 21:29:31,736 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520571386_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:31,736 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520571386_0001_00_000002 completed but with errors
2016-05-17 21:29:31 Completed Dag: dag_1463520571386_0001_1
2016-05-17 21:29:31,741 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520571386_0001_1, dagState=FAILED
2016-05-17 21:29:31,741 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520571386_0001_1_post
2016-05-17 21:29:31,741 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:31,741 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:31,742 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571386_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:31,742 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520571386_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:31,742 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571386_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:31,743 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520571386_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:31,744 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/-1_VALUES_ALL, applicationId=application_1463520571386_0001
2016-05-17 21:29:31,751 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:31,751 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:31,751 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:31,751 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:31,751 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:31,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:31,752 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:31,752 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:31,752 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:31,752 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:31,752 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:31,753 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:31,753 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:31,757 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/6EFD18ADFC9945E6A363DDB8370EC793/C6C6CDF8B36C43F49207E25C0578AE52/.tez/application_1463520571386_0001
2016-05-17 21:29:31,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:31,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...blyplatform/-1_VALUES_ALL
2016-05-17 21:29:31,757 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:31,758 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_ALL/_temporary
2016-05-17 21:29:31,758 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.390
2016-05-17 21:29:31,758 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_ALL
2016-05-17 21:29:31,779 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:31,779 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:31,811 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 21:29:31,821 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-17 21:29:31,821 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:31,821 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:31,821 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:31,836 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:31,843 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_REPLACE
2016-05-17 21:29:31,900 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:31,900 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:31,900 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_REPLACE"]
2016-05-17 21:29:31,900 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:31,900 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:31,900 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:31,901 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...latform/-1_VALUES_REPLACE
2016-05-17 21:29:31,901 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:31,913 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:31,917 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:31,918 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:31,918 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:31,929 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7146B970528447BC82397861FCD8F093/E531CBD2B46D43CE9C62B6897895DC11/.tez/application_1463520571917_0001 doesn't exist and is created
2016-05-17 21:29:31,946 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:31,946 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:31,947 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7146B970528447BC82397861FCD8F093/E531CBD2B46D43CE9C62B6897895DC11/.tez/application_1463520571917_0001
2016-05-17 21:29:31,949 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520571917_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:31,959 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:31,960 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:31,961 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:31,961 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:31,961 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7146B970528447BC82397861FCD8F093/E531CBD2B46D43CE9C62B6897895DC11/.tez/application_1463520571917_0001/localmode-log-dir
2016-05-17 21:29:31,962 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7146B970528447BC82397861FCD8F093/E531CBD2B46D43CE9C62B6897895DC11/.tez/application_1463520571917_0001/localmode-log-dir/history.txt.appattempt_1463520571917_0001_000000, maxErrors=10
2016-05-17 21:29:31,962 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:31,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520571917_0001_000000, appSubmitTime=1463520571949, launchTime=1463520571949
2016-05-17 21:29:31,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:31,967 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:31,968 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:31,968 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:31,969 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34296
2016-05-17 21:29:31,970 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:31,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520571917_0001_000000, startTime=1463520571974
2016-05-17 21:29:31,975 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:31,975 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:32,046 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:32,047 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:32,047 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_REPLACE, applicationId=application_1463520571917_0001, dagName=[7146B970528447BC82397861FCD8F093/E531CBD2B46D43CE9C62B6897895DC11] every/(1/1) ...latform/-1_VALUES_REPLACE
2016-05-17 21:29:32,062 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7146B970528447BC82397861FCD8F093/E531CBD2B46D43CE9C62B6897895DC11] every/(1/1) ...latform/-1_VALUES_REPLACE
2016-05-17 21:29:32,064 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520571917_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7146B970528447BC82397861FCD8F093/E531CBD2B46D43CE9C62B6897895DC11/.tez/application_1463520571917_0001/localmode-log-dir/dag_1463520571917_0001_1.dot
2016-05-17 21:29:32,064 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7146B970528447BC82397861FCD8F093/E531CBD2B46D43CE9C62B6897895DC11/.tez/application_1463520571917_0001/localmode-log-dir/dag_1463520571917_0001_1-tez-dag.pb.txt
2016-05-17 21:29:32,073 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520571917_0001_1
2016-05-17 21:29:32,073 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:32,073 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7146B970528447BC82397861FCD8F093/E531CBD2B46D43CE9C62B6897895DC11] every/(1/1) ...latform/-1_VALUES_REPLACE
2016-05-17 21:29:32 Running Dag: dag_1463520571917_0001_1
2016-05-17 21:29:32,084 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520571917_0001_1, submitTime=1463520572062
2016-05-17 21:29:32,084 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:32,085 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FF4CC374BFB84E58B45E12632C399368
2016-05-17 21:29:32,085 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:32,086 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:32,086 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520571917_0001_1, initTime=1463520572085
2016-05-17 21:29:32,086 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520571917_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:32,087 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:DAG_STARTED]: dagID=dag_1463520571917_0001_1, startTime=1463520572087
2016-05-17 21:29:32,087 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:32,087 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520571917_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:32,087 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_REPLACE, applicationId=application_1463520571917_0001, dagName=[7146B970528447BC82397861FCD8F093/E531CBD2B46D43CE9C62B6897895DC11] every/(1/1) ...latform/-1_VALUES_REPLACE
2016-05-17 21:29:32,087 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: ADDC695299ED4017BF524B7893CB9895 : {D7EF3EF651D7471A80C0D821F9815AEB={InputName=D7EF3EF651D7471A80C0D821F9815AEB}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:32,088 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: D7EF3EF651D7471A80C0D821F9815AEB, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:32,088 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895]
2016-05-17 21:29:32,088 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895]
2016-05-17 21:29:32,088 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895]
2016-05-17 21:29:32,088 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895]. Starting root input initializers: 1
2016-05-17 21:29:32,089 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:32,089 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:32,089 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368]
2016-05-17 21:29:32,090 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: D7EF3EF651D7471A80C0D821F9815AEB on vertex vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895]
2016-05-17 21:29:32,091 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:32,092 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:32,092 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368]
2016-05-17 21:29:32,092 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368]
2016-05-17 21:29:32,092 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368]
2016-05-17 21:29:32,093 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=C9B8171B177044C0928AFE4F0CC19D40, vertexId=vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:32,093 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=C9B8171B177044C0928AFE4F0CC19D40, vertexId=vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368]
2016-05-17 21:29:32,092 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input D7EF3EF651D7471A80C0D821F9815AEB asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:32,094 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:32,094 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:32,095 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=C9B8171B177044C0928AFE4F0CC19D40, vertexName=FF4CC374BFB84E58B45E12632C399368, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:32,095 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=C9B8171B177044C0928AFE4F0CC19D40, vertexId=vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368]
2016-05-17 21:29:32,096 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:32,096 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:32,097 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: D7EF3EF651D7471A80C0D821F9815AEB on vertex vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895]
2016-05-17 21:29:32,097 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FF4CC374BFB84E58B45E12632C399368, vertexId=vertex_1463520571917_0001_1_01, initRequestedTime=1463520572089, initedTime=1463520572096, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:32,097 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:32,098 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:32,098 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {D7EF3EF651D7471A80C0D821F9815AEB=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:32,098 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895] parallelism set to 1
2016-05-17 21:29:32,098 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:32,099 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520571917_0001_1_00, eventCount=1
2016-05-17 21:29:32,099 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:32,099 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ADDC695299ED4017BF524B7893CB9895, vertexId=vertex_1463520571917_0001_1_00, initRequestedTime=1463520572087, initedTime=1463520572099, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:32,099 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895] with distanceFromRoot: 0
2016-05-17 21:29:32,100 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:32,100 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in ADDC695299ED4017BF524B7893CB9895
2016-05-17 21:29:32,100 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895]
2016-05-17 21:29:32,100 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:32,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520571917_0001_1_00, startRequestedTime=1463520572100, startedTime=1463520572100
2016-05-17 21:29:32,101 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:32,101 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:32,102 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:TASK_STARTED]: vertexName=ADDC695299ED4017BF524B7893CB9895, taskId=task_1463520571917_0001_1_00_000000, scheduledTime=1463520572101, launchTime=1463520572101
2016-05-17 21:29:32,102 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520571917_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:32,102 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520571917_0001_1_00 for vertex: vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368] numStartedSources: 1 numSources: 1
2016-05-17 21:29:32,102 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368] with distanceFromRoot: 1
2016-05-17 21:29:32,102 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895]
2016-05-17 21:29:32,103 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: ADDC695299ED4017BF524B7893CB9895 in vertex: FF4CC374BFB84E58B45E12632C399368
2016-05-17 21:29:32,103 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FF4CC374BFB84E58B45E12632C399368 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:32,103 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:32,104 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520571917_0001_1_01, startRequestedTime=1463520572103, startedTime=1463520572103
2016-05-17 21:29:32,104 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:32,104 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7146B970528447BC82397861FCD8F093/E531CBD2B46D43CE9C62B6897895DC11] every/(1/1) ...latform/-1_VALUES_REPLACE, VertexName: ADDC695299ED4017BF524B7893CB9895, VertexParallelism: 1, TaskAttemptID:attempt_1463520571917_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D7EF3EF651D7471A80C0D821F9815AEB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FF4CC374BFB84E58B45E12632C399368, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:32,104 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520571917_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:32,105 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:32,105 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:32,106 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:32,106 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:32,106 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520571917_0001_00_000001
2016-05-17 21:29:32,106 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571917_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:32,106 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520571917_0001, with dag id: dag_1463520571917_0001_1
2016-05-17 21:29:32,110 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520571917_0001_00_000001
2016-05-17 21:29:32,110 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:32,111 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571917_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:32,111 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:32,111 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520571917_0001_00_000001, launchTime=1463520572111
2016-05-17 21:29:32,113 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520571917_0001_00_000001
2016-05-17 21:29:32,113 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520571917_0001_1_00_000000_0] to container: [container_1463520571917_0001_00_000001]
2016-05-17 21:29:32,113 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571917_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:32,114 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520571917_0001_00_000001 given task: attempt_1463520571917_0001_1_00_000000_0
2016-05-17 21:29:32,115 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520571917_0001_1_00_000000_0] started. Is using containerId: [container_1463520571917_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:32,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ADDC695299ED4017BF524B7893CB9895, taskAttemptId=attempt_1463520571917_0001_1_00_000000_0, startTime=1463520572115, containerId=container_1463520571917_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520571917_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:32,115 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520571917_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:32,115 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520571917_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:32,115 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:32,115 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520571917_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520571917_0001_1_00_000000_0
2016-05-17 21:29:32 Starting to run new task attempt: attempt_1463520571917_0001_1_00_000000_0
2016-05-17 21:29:32,117 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520571917_0001_1_00_000000_0
2016-05-17 21:29:32,117 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:32,117 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:32,117 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:32,117 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:32,117 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:32,118 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:32,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7146B970528447BC82397861FCD8F093/E531CBD2B46D43CE9C62B6897895DC11] every/(1/1) ...latform/-1_VALUES_REPLACE, VertexName: ADDC695299ED4017BF524B7893CB9895, VertexParallelism: 1, TaskAttemptID:attempt_1463520571917_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D7EF3EF651D7471A80C0D821F9815AEB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FF4CC374BFB84E58B45E12632C399368, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:32,118 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:32,118 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520571917_0001_1_00_000000_0
2016-05-17 21:29:32,119 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:32,119 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:32,120 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D7EF3EF651D7471A80C0D821F9815AEB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:32,119 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520571917_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:32,120 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FF4CC374BFB84E58B45E12632C399368, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:32,120 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:32,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:32,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D7EF3EF651D7471A80C0D821F9815AEB
2016-05-17 21:29:32,121 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: D7EF3EF651D7471A80C0D821F9815AEB
2016-05-17 21:29:32,123 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:32,123 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: D7EF3EF651D7471A80C0D821F9815AEB
2016-05-17 21:29:32,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D7EF3EF651D7471A80C0D821F9815AEB
2016-05-17 21:29:32,123 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:32,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:32,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FF4CC374BFB84E58B45E12632C399368
2016-05-17 21:29:32,128 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:32,128 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FF4CC374BFB84E58B45E12632C399368
2016-05-17 21:29:32,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FF4CC374BFB84E58B45E12632C399368
2016-05-17 21:29:32,129 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ADDC695299ED4017BF524B7893CB9895, ordinal: 0
2016-05-17 21:29:32,129 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADDC695299ED4017BF524B7893CB9895, mem on start (mb), free: 276, total: 360, max: 672
2016-05-17 21:29:32,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:32,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:32,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:32,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:32,130 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:32,130 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:32,130 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:32,131 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:32,131 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:32,131 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:32,131 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D7EF3EF651D7471A80C0D821F9815AEB, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:32,131 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FF4CC374BFB84E58B45E12632C399368, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:32,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:32,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D7EF3EF651D7471A80C0D821F9815AEB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:32,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:32,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:32,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D7EF3EF651D7471A80C0D821F9815AEB
2016-05-17 21:29:32,133 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:32,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D7EF3EF651D7471A80C0D821F9815AEB
2016-05-17 21:29:32,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:32,133 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520571917_0001_1_00_000000_0
2016-05-17 21:29:32,134 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:32,134 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@135d6cb1
2016-05-17 21:29:32,135 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@135d6cb1
2016-05-17 21:29:32,135 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:32,136 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:32,136 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:32,137 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: D7EF3EF651D7471A80C0D821F9815AEB
2016-05-17 21:29:32,137 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 0CAE8AD1569942F1BAFB9FCE88BBC72D
2016-05-17 21:29:32,137 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 0CAE8AD1569942F1BAFB9FCE88BBC72D
2016-05-17 21:29:32,138 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:32,141 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:32,141 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@679280dc; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@434a20a3; comparator=cascading.tuple.hadoop.util.TupleComparator@37a4cfd8; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:32,141 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:32,141 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:32,141 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:32,154 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:32,154 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:32,155 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:32,155 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:32,155 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:32,156 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:32,156 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:32,156 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ADDC695299ED4017BF524B7893CB9895, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:32,159 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520571917_0001_1_00_000000_0
2016-05-17 21:29:32,159 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADDC695299ED4017BF524B7893CB9895, mem on close (mb), free: 173, total: 360, max: 672
2016-05-17 21:29:32,159 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:32,159 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:32,159 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:32,160 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:32,165 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:32 Completed running task attempt: attempt_1463520571917_0001_1_00_000000_0
2016-05-17 21:29:32,168 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520571917_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:32,169 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520572156, Process_Duration=3, Process_End_Time=1463520572159, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:32,169 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520571917_0001_1_00, eventCount=1
2016-05-17 21:29:32,169 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:32,169 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:32,169 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520571917_0001_00_000001
2016-05-17 21:29:32,170 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ADDC695299ED4017BF524B7893CB9895, taskAttemptId=attempt_1463520571917_0001_1_00_000000_0, startTime=1463520572115, finishTime=1463520572169, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520572156, Process_Duration=3, Process_End_Time=1463520572159, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:32,170 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:32,170 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520571917_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:32,171 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520571917_0001_1_00_000000_0
2016-05-17 21:29:32,171 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:TASK_FINISHED]: vertexName=ADDC695299ED4017BF524B7893CB9895, taskId=task_1463520571917_0001_1_00_000000, startTime=1463520572115, finishTime=1463520572171, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1463520571917_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520572156, Process_Duration=3, Process_End_Time=1463520572159, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:32,172 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520571917_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:32,172 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:32,172 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:32,172 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895] : 1
2016-05-17 21:29:32,172 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895]
2016-05-17 21:29:32,174 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:VERTEX_FINISHED]: vertexName=ADDC695299ED4017BF524B7893CB9895, vertexId=vertex_1463520571917_0001_1_00, initRequestedTime=1463520572087, initedTime=1463520572099, startRequestedTime=1463520572100, startedTime=1463520572100, finishTime=1463520572173, timeTaken=73, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520572156, Process_Duration=3, Process_End_Time=1463520572159, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520572115, firstTasksToStart=[ task_1463520571917_0001_1_00_000000 ], lastTaskFinishTime=1463520572169, lastTasksToFinish=[ task_1463520571917_0001_1_00_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520571917_0001_1_00_000000 ], longestDurationTasks=[ task_1463520571917_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:32,174 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:32,174 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:32,174 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571917_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:32,175 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368] attempt: attempt_1463520571917_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:32,175 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FF4CC374BFB84E58B45E12632C399368
2016-05-17 21:29:32,175 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368]
2016-05-17 21:29:32,175 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520571917_0001_00_000001, stoppedTime=1463520572175, exitStatus=0
2016-05-17 21:29:32,175 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571917_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:32,175 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:32,176 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:TASK_STARTED]: vertexName=FF4CC374BFB84E58B45E12632C399368, taskId=task_1463520571917_0001_1_01_000000, scheduledTime=1463520572176, launchTime=1463520572176
2016-05-17 21:29:32,176 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:32,176 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:32,176 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520571917_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:32,176 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520571917_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:32,177 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368]
2016-05-17 21:29:32,177 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571917_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:32,177 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7146B970528447BC82397861FCD8F093/E531CBD2B46D43CE9C62B6897895DC11] every/(1/1) ...latform/-1_VALUES_REPLACE, VertexName: FF4CC374BFB84E58B45E12632C399368, VertexParallelism: 1, TaskAttemptID:attempt_1463520571917_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ADDC695299ED4017BF524B7893CB9895, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C9B8171B177044C0928AFE4F0CC19D40, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:32,177 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520571917_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:32,177 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:32,178 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520571917_0001_00_000002
2016-05-17 21:29:32,178 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571917_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:32,179 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520571917_0001_00_000002
2016-05-17 21:29:32,179 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:32,180 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571917_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:32,180 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520571917_0001_00_000002, launchTime=1463520572180
2016-05-17 21:29:32,180 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520571917_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:32,180 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:32,181 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520571917_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:32,181 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520571917_0001_00_000001, Exiting
2016-05-17 21:29:32,181 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520571917_0001_00_000002
2016-05-17 21:29:32,181 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520571917_0001_00_000001
2016-05-17 21:29:32,182 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520571917_0001_00_000001
2016-05-17 21:29:32,182 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520571917_0001_1_01_000000_0] to container: [container_1463520571917_0001_00_000002]
2016-05-17 21:29:32,182 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571917_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:32,182 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520571917_0001_00_000002 given task: attempt_1463520571917_0001_1_01_000000_0
2016-05-17 21:29:32,182 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520571917_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520571917_0001_1_01_000000_0
2016-05-17 21:29:32,183 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520571917_0001_1_01_000000_0] started. Is using containerId: [container_1463520571917_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:32,183 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:32,183 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FF4CC374BFB84E58B45E12632C399368, taskAttemptId=attempt_1463520571917_0001_1_01_000000_0, startTime=1463520572183, containerId=container_1463520571917_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520571917_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:32,183 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520571917_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:32,184 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520571917_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:32 Starting to run new task attempt: attempt_1463520571917_0001_1_01_000000_0
2016-05-17 21:29:32,184 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520571917_0001_1_01_000000_0
2016-05-17 21:29:32,184 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:32,185 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:32,185 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:32,185 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:32,185 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:32,185 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:32,185 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7146B970528447BC82397861FCD8F093/E531CBD2B46D43CE9C62B6897895DC11] every/(1/1) ...latform/-1_VALUES_REPLACE, VertexName: FF4CC374BFB84E58B45E12632C399368, VertexParallelism: 1, TaskAttemptID:attempt_1463520571917_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ADDC695299ED4017BF524B7893CB9895, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C9B8171B177044C0928AFE4F0CC19D40, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:32,185 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:32,184 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520571917_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:32,184 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520571917_0001_00_000001 completed successfully
2016-05-17 21:29:32,186 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520571917_0001_1_01_000000_0
2016-05-17 21:29:32,186 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:32,186 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571917_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:32,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:32,189 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520571917_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:32,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=ADDC695299ED4017BF524B7893CB9895, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:32,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:32,190 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:32,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: ADDC695299ED4017BF524B7893CB9895
2016-05-17 21:29:32,190 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C9B8171B177044C0928AFE4F0CC19D40, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:32,192 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:32,192 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C9B8171B177044C0928AFE4F0CC19D40
2016-05-17 21:29:32,192 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:32,192 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: C9B8171B177044C0928AFE4F0CC19D40
2016-05-17 21:29:32,193 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:32,193 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:32,193 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: ADDC695299ED4017BF524B7893CB9895
2016-05-17 21:29:32,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: ADDC695299ED4017BF524B7893CB9895
2016-05-17 21:29:32,195 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:32,203 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FF4CC374BFB84E58B45E12632C399368, ordinal: 1
2016-05-17 21:29:32,204 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF4CC374BFB84E58B45E12632C399368, mem on start (mb), free: 166, total: 360, max: 672
2016-05-17 21:29:32,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:32,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:32,205 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:32,206 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:32,206 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C9B8171B177044C0928AFE4F0CC19D40
2016-05-17 21:29:32,206 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:32,206 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:32,206 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:32,206 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:32,207 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:32,208 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:32,208 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:32,208 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C9B8171B177044C0928AFE4F0CC19D40, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:32,208 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, ADDC695299ED4017BF524B7893CB9895, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:32,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:32,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: ADDC695299ED4017BF524B7893CB9895 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:32,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: ADDC695299ED4017BF524B7893CB9895
2016-05-17 21:29:32,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:32,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:32,210 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:32,210 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:32,210 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: ADDC695299ED4017BF524B7893CB9895 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:32,210 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:32,211 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:32,211 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:32,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: ADDC695299ED4017BF524B7893CB9895
2016-05-17 21:29:32,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:32,213 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520571917_0001_1_01_000000_0
2016-05-17 21:29:32,213 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520571917_0001_1_00_000000_0_10081, runDuration: 40191, ]
2016-05-17 21:29:32,215 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520571917_0001_1_00_000000_0_10081] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520572215, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:32,215 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:32,215 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:32,215 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : ADDC695299ED4017BF524B7893CB9895
2016-05-17 21:29:32,215 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:32,215 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:32,216 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [ADDC695299ED4017BF524B7893CB9895] #14 in 2ms
2016-05-17 21:29:32,216 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [ADDC695299ED4017BF524B7893CB9895] #14, status:false, isInterrupted:false
2016-05-17 21:29:32,216 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 0CAE8AD1569942F1BAFB9FCE88BBC72D
2016-05-17 21:29:32,216 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:32,217 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_REPLACE"], id: C9B8171B177044C0928AFE4F0CC19D40
2016-05-17 21:29:32,218 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_REPLACE"]
2016-05-17 21:29:32,218 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 0CAE8AD1569942F1BAFB9FCE88BBC72D, for 1 inputs
2016-05-17 21:29:32,218 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:32,218 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:32,219 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:32,219 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:32,219 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:32,219 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : ADDC695299ED4017BF524B7893CB9895
2016-05-17 21:29:32,219 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:32,220 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FF4CC374BFB84E58B45E12632C399368, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:29:32,221 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:32,222 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@23516313; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@343f8682
2016-05-17 21:29:32,223 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:32,227 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:32,232 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:32,232 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:32,232 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520571917_0001_1_01_000000_0
2016-05-17 21:29:32,233 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520571917_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:32,233 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520571917_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:32,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FF4CC374BFB84E58B45E12632C399368, taskAttemptId=attempt_1463520571917_0001_1_01_000000_0, startTime=1463520572183, finishTime=1463520572238, timeTaken=55, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520572220, Process_Duration=13, Process_End_Time=1463520572233, Read_Duration=0, Tuples_Read=1
2016-05-17 21:29:32,239 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520571917_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:32,239 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520571917_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:32,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:TASK_FINISHED]: vertexName=FF4CC374BFB84E58B45E12632C399368, taskId=task_1463520571917_0001_1_01_000000, startTime=1463520572183, finishTime=1463520572239, timeTaken=56, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:32,240 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520571917_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:32,239 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:32,240 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368] : 1
2016-05-17 21:29:32,240 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368] because task failed: task_1463520571917_0001_1_01_000000
2016-05-17 21:29:32,240 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:32,240 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:32,240 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368]
2016-05-17 21:29:32,241 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=C9B8171B177044C0928AFE4F0CC19D40, vertexId=vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368]
2016-05-17 21:29:32,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:32,241 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:32,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:VERTEX_FINISHED]: vertexName=FF4CC374BFB84E58B45E12632C399368, vertexId=vertex_1463520571917_0001_1_01, initRequestedTime=1463520572089, initedTime=1463520572096, startRequestedTime=1463520572103, startedTime=1463520572103, finishTime=1463520572242, timeTaken=139, status=FAILED, diagnostics=Task failed, taskId=task_1463520571917_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520572183, firstTasksToStart=[ task_1463520571917_0001_1_01_000000 ], lastTaskFinishTime=1463520572238, lastTasksToFinish=[ task_1463520571917_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:32,242 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:32,242 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:32,243 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520571917_0001_00_000002, stoppedTime=1463520572243, exitStatus=0
2016-05-17 21:29:32,243 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571917_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:32,243 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520571917_0001_1_01_000000_0 failed: true container: container_1463520571917_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:32,244 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:32,244 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:32,244 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520571917_0001_1
2016-05-17 21:29:32,244 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520571917_0001_1_00 [ADDC695299ED4017BF524B7893CB9895]
2016-05-17 21:29:32,244 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: C9B8171B177044C0928AFE4F0CC19D40 for vertex: vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368]
2016-05-17 21:29:32,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520572220, Process_Duration=13, Process_End_Time=1463520572233, Read_Duration=0, Tuples_Read=1]]
2016-05-17 21:29:32,245 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF4CC374BFB84E58B45E12632C399368, mem on close (mb), free: 159, total: 360, max: 672
2016-05-17 21:29:32,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=FF4CC374BFB84E58B45E12632C399368, index=1
2016-05-17 21:29:32,245 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:32,245 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: ADDC695299ED4017BF524B7893CB9895
2016-05-17 21:29:32,246 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:32,246 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520571917_0001_1, queueSize=0
2016-05-17 21:29:32,247 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:32,248 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520571917_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520571917_0001_1, startTime=1463520572087, finishTime=1463520572244, timeTaken=157, status=FAILED, diagnostics=Vertex failed, vertexName=FF4CC374BFB84E58B45E12632C399368, vertexId=vertex_1463520571917_0001_1_01, diagnostics=[Task failed, taskId=task_1463520571917_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520571917_0001_1_01 [FF4CC374BFB84E58B45E12632C399368] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520572156, Process_Duration=3, Process_End_Time=1463520572159, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:32,248 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520571917_0001_1 finished with state: FAILED
2016-05-17 21:29:32,248 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520571917_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:32,248 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520571917_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:32 Completed Dag: dag_1463520571917_0001_1
2016-05-17 21:29:32,249 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:32,249 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:32,249 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:32,249 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:32,250 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:32,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=FF4CC374BFB84E58B45E12632C399368, sourceVertex=ADDC695299ED4017BF524B7893CB9895
2016-05-17 21:29:32,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=FF4CC374BFB84E58B45E12632C399368, sourceVertex=C9B8171B177044C0928AFE4F0CC19D40
2016-05-17 21:29:32,249 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520571917_0001_1, dagState=FAILED
2016-05-17 21:29:32,251 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520571917_0001_1_post
2016-05-17 21:29:32,251 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:32,251 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:32,251 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520571917_0001_00_000002
2016-05-17 21:29:32,251 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520571917_0001_00_000002
2016-05-17 21:29:32,252 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520571917_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:32,252 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520571917_0001_00_000002 completed but with errors
2016-05-17 21:29:32,252 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571917_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:32,253 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520571917_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:32,253 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520571917_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:32,254 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520571917_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:32,256 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/-1_VALUES_REPLACE, applicationId=application_1463520571917_0001
2016-05-17 21:29:32,258 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:32,258 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:32,258 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:32,258 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:32,258 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:32,259 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:32,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:32,260 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:32,260 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:32,260 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:32,260 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:32,260 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:32,260 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:32,265 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7146B970528447BC82397861FCD8F093/E531CBD2B46D43CE9C62B6897895DC11/.tez/application_1463520571917_0001
2016-05-17 21:29:32,266 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:32,266 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...latform/-1_VALUES_REPLACE
2016-05-17 21:29:32,266 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:32,267 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_REPLACE/_temporary
2016-05-17 21:29:32,267 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.366
2016-05-17 21:29:32,267 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_REPLACE
2016-05-17 21:29:32,290 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:32,290 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:32,332 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 21:29:32,338 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-05-17 21:29:32,338 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:32,339 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:32,339 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:32,359 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:32,366 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_SWAP
2016-05-17 21:29:32,410 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:32,411 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:32,411 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_SWAP"]
2016-05-17 21:29:32,411 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:32,411 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:32,411 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:32,412 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...lyplatform/-1_VALUES_SWAP
2016-05-17 21:29:32,412 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:32,443 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:32,448 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:32,448 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:32,448 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:32,457 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/42A5BEB476A74110AD5A51062150BABF/FF91F6BDB6E24CC4A39FF5C91AE2D593/.tez/application_1463520572448_0001 doesn't exist and is created
2016-05-17 21:29:32,472 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:32,473 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:32,473 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/42A5BEB476A74110AD5A51062150BABF/FF91F6BDB6E24CC4A39FF5C91AE2D593/.tez/application_1463520572448_0001
2016-05-17 21:29:32,475 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520572448_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:32,485 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:32,485 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:32,486 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:32,487 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:32,487 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/42A5BEB476A74110AD5A51062150BABF/FF91F6BDB6E24CC4A39FF5C91AE2D593/.tez/application_1463520572448_0001/localmode-log-dir
2016-05-17 21:29:32,487 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/42A5BEB476A74110AD5A51062150BABF/FF91F6BDB6E24CC4A39FF5C91AE2D593/.tez/application_1463520572448_0001/localmode-log-dir/history.txt.appattempt_1463520572448_0001_000000, maxErrors=10
2016-05-17 21:29:32,487 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:32,487 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520572448_0001_000000, appSubmitTime=1463520572475, launchTime=1463520572475
2016-05-17 21:29:32,489 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:32,492 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:32,496 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:32,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:32,498 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56450
2016-05-17 21:29:32,499 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:32,502 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520572448_0001_000000, startTime=1463520572502
2016-05-17 21:29:32,502 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:32,502 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:32,573 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:32,573 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:32,573 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/-1_VALUES_SWAP, applicationId=application_1463520572448_0001, dagName=[42A5BEB476A74110AD5A51062150BABF/FF91F6BDB6E24CC4A39FF5C91AE2D593] every/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-05-17 21:29:32,587 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [42A5BEB476A74110AD5A51062150BABF/FF91F6BDB6E24CC4A39FF5C91AE2D593] every/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-05-17 21:29:32,589 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520572448_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/42A5BEB476A74110AD5A51062150BABF/FF91F6BDB6E24CC4A39FF5C91AE2D593/.tez/application_1463520572448_0001/localmode-log-dir/dag_1463520572448_0001_1.dot
2016-05-17 21:29:32,589 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/42A5BEB476A74110AD5A51062150BABF/FF91F6BDB6E24CC4A39FF5C91AE2D593/.tez/application_1463520572448_0001/localmode-log-dir/dag_1463520572448_0001_1-tez-dag.pb.txt
2016-05-17 21:29:32,596 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520572448_0001_1
2016-05-17 21:29:32,596 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:32,596 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [42A5BEB476A74110AD5A51062150BABF/FF91F6BDB6E24CC4A39FF5C91AE2D593] every/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-05-17 21:29:32 Running Dag: dag_1463520572448_0001_1
2016-05-17 21:29:32,607 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520572448_0001_1, submitTime=1463520572588
2016-05-17 21:29:32,607 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:32,607 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F2CE52C638834D82A98E13067E2582FE
2016-05-17 21:29:32,608 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:32,608 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:32,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520572448_0001_1, initTime=1463520572607
2016-05-17 21:29:32,608 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520572448_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:32,609 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:DAG_STARTED]: dagID=dag_1463520572448_0001_1, startTime=1463520572609
2016-05-17 21:29:32,609 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:32,609 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:32,609 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520572448_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:32,610 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A997EC81807D4A2585129DA7F1C5BD5B : {E1215D37A27C422C80064D640811510B={InputName=E1215D37A27C422C80064D640811510B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:32,610 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E1215D37A27C422C80064D640811510B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:32,610 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B]
2016-05-17 21:29:32,610 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B]
2016-05-17 21:29:32,610 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B]
2016-05-17 21:29:32,610 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B]. Starting root input initializers: 1
2016-05-17 21:29:32,611 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/-1_VALUES_SWAP, applicationId=application_1463520572448_0001, dagName=[42A5BEB476A74110AD5A51062150BABF/FF91F6BDB6E24CC4A39FF5C91AE2D593] every/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-05-17 21:29:32,611 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:32,611 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE]
2016-05-17 21:29:32,614 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:32,614 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE]
2016-05-17 21:29:32,614 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE]
2016-05-17 21:29:32,614 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE]
2016-05-17 21:29:32,614 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=E9E72B71C87E478AA80F66EBA3620D0F, vertexId=vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:32,615 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=E9E72B71C87E478AA80F66EBA3620D0F, vertexId=vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE]
2016-05-17 21:29:32,616 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=E9E72B71C87E478AA80F66EBA3620D0F, vertexName=F2CE52C638834D82A98E13067E2582FE, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:32,616 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E1215D37A27C422C80064D640811510B on vertex vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B]
2016-05-17 21:29:32,616 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=E9E72B71C87E478AA80F66EBA3620D0F, vertexId=vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE]
2016-05-17 21:29:32,618 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F2CE52C638834D82A98E13067E2582FE, vertexId=vertex_1463520572448_0001_1_01, initRequestedTime=1463520572611, initedTime=1463520572618, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:32,618 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:32,618 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:32,620 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:32,620 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E1215D37A27C422C80064D640811510B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:32,621 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:32,621 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:32,622 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:32,622 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E1215D37A27C422C80064D640811510B on vertex vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B]
2016-05-17 21:29:32,623 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:32,623 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E1215D37A27C422C80064D640811510B=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:32,623 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B] parallelism set to 1
2016-05-17 21:29:32,623 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:32,623 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520572448_0001_1_00, eventCount=1
2016-05-17 21:29:32,624 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:32,624 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A997EC81807D4A2585129DA7F1C5BD5B, vertexId=vertex_1463520572448_0001_1_00, initRequestedTime=1463520572610, initedTime=1463520572624, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:32,624 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B] with distanceFromRoot: 0
2016-05-17 21:29:32,624 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:32,624 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A997EC81807D4A2585129DA7F1C5BD5B
2016-05-17 21:29:32,625 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B]
2016-05-17 21:29:32,625 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:32,625 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520572448_0001_1_00, startRequestedTime=1463520572624, startedTime=1463520572624
2016-05-17 21:29:32,625 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:32,626 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:32,626 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:TASK_STARTED]: vertexName=A997EC81807D4A2585129DA7F1C5BD5B, taskId=task_1463520572448_0001_1_00_000000, scheduledTime=1463520572626, launchTime=1463520572626
2016-05-17 21:29:32,626 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520572448_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:32,626 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520572448_0001_1_00 for vertex: vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE] numStartedSources: 1 numSources: 1
2016-05-17 21:29:32,626 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE] with distanceFromRoot: 1
2016-05-17 21:29:32,627 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B]
2016-05-17 21:29:32,627 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A997EC81807D4A2585129DA7F1C5BD5B in vertex: F2CE52C638834D82A98E13067E2582FE
2016-05-17 21:29:32,627 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F2CE52C638834D82A98E13067E2582FE with 1 source tasks and 1 pending tasks
2016-05-17 21:29:32,627 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:32,627 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520572448_0001_1_01, startRequestedTime=1463520572627, startedTime=1463520572627
2016-05-17 21:29:32,628 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:32,628 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [42A5BEB476A74110AD5A51062150BABF/FF91F6BDB6E24CC4A39FF5C91AE2D593] every/(1/1) ...lyplatform/-1_VALUES_SWAP, VertexName: A997EC81807D4A2585129DA7F1C5BD5B, VertexParallelism: 1, TaskAttemptID:attempt_1463520572448_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E1215D37A27C422C80064D640811510B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F2CE52C638834D82A98E13067E2582FE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:32,628 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520572448_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:32,628 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:32,629 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:32,629 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:32,629 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:32,630 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520572448_0001_00_000001
2016-05-17 21:29:32,630 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572448_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:32,632 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520572448_0001_00_000001
2016-05-17 21:29:32,632 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:32,633 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:32,633 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520572448_0001_00_000001, launchTime=1463520572633
2016-05-17 21:29:32,633 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572448_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:32,634 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520572448_0001_00_000001
2016-05-17 21:29:32,634 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520572448_0001_1_00_000000_0] to container: [container_1463520572448_0001_00_000001]
2016-05-17 21:29:32,634 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572448_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:32,635 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520572448_0001_1_00_000000_0] started. Is using containerId: [container_1463520572448_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:32,635 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A997EC81807D4A2585129DA7F1C5BD5B, taskAttemptId=attempt_1463520572448_0001_1_00_000000_0, startTime=1463520572635, containerId=container_1463520572448_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520572448_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:32,635 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:32,635 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520572448_0001_00_000001 given task: attempt_1463520572448_0001_1_00_000000_0
2016-05-17 21:29:32,635 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520572448_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:32,636 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520572448_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:32,636 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520572448_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520572448_0001_1_00_000000_0
2016-05-17 21:29:32 Starting to run new task attempt: attempt_1463520572448_0001_1_00_000000_0
2016-05-17 21:29:32,636 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520572448_0001_1_00_000000_0
2016-05-17 21:29:32,636 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:32,636 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520572448_0001, with dag id: dag_1463520572448_0001_1
2016-05-17 21:29:32,637 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:32,637 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:32,638 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:32,638 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:32,638 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:32,638 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [42A5BEB476A74110AD5A51062150BABF/FF91F6BDB6E24CC4A39FF5C91AE2D593] every/(1/1) ...lyplatform/-1_VALUES_SWAP, VertexName: A997EC81807D4A2585129DA7F1C5BD5B, VertexParallelism: 1, TaskAttemptID:attempt_1463520572448_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E1215D37A27C422C80064D640811510B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F2CE52C638834D82A98E13067E2582FE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:32,638 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:32,639 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520572448_0001_1_00_000000_0
2016-05-17 21:29:32,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:32,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:32,641 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520572448_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:32,641 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E1215D37A27C422C80064D640811510B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:32,642 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:32,642 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E1215D37A27C422C80064D640811510B
2016-05-17 21:29:32,642 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E1215D37A27C422C80064D640811510B
2016-05-17 21:29:32,643 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:32,643 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F2CE52C638834D82A98E13067E2582FE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:32,643 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:32,644 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F2CE52C638834D82A98E13067E2582FE
2016-05-17 21:29:32,645 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:32,645 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F2CE52C638834D82A98E13067E2582FE
2016-05-17 21:29:32,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F2CE52C638834D82A98E13067E2582FE
2016-05-17 21:29:32,646 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:32,647 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:32,647 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E1215D37A27C422C80064D640811510B
2016-05-17 21:29:32,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E1215D37A27C422C80064D640811510B
2016-05-17 21:29:32,653 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A997EC81807D4A2585129DA7F1C5BD5B, ordinal: 0
2016-05-17 21:29:32,653 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A997EC81807D4A2585129DA7F1C5BD5B, mem on start (mb), free: 283, total: 356, max: 672
2016-05-17 21:29:32,653 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:32,653 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:32,653 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:32,653 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:32,654 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:32,654 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:32,654 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:32,654 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:32,655 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:32,655 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:32,655 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E1215D37A27C422C80064D640811510B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:32,655 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F2CE52C638834D82A98E13067E2582FE, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:32,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:32,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E1215D37A27C422C80064D640811510B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:32,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E1215D37A27C422C80064D640811510B
2016-05-17 21:29:32,656 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:32,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E1215D37A27C422C80064D640811510B
2016-05-17 21:29:32,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:32,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:32,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:32,657 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520572448_0001_1_00_000000_0
2016-05-17 21:29:32,657 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:32,658 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@76416019
2016-05-17 21:29:32,659 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@76416019
2016-05-17 21:29:32,659 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:32,659 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:32,659 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:32,660 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E1215D37A27C422C80064D640811510B
2016-05-17 21:29:32,660 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 66E1CDF71F16435FBB7ADC00BE848245
2016-05-17 21:29:32,660 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 66E1CDF71F16435FBB7ADC00BE848245
2016-05-17 21:29:32,662 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:32,664 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:32,664 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@4de2c090; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@61d055e2; comparator=cascading.tuple.hadoop.util.TupleComparator@34b69a99; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:32,664 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:32,664 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:32,665 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:32,680 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:32,680 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:32,681 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:32,681 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:32,681 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:32,682 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:32,683 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:32,683 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A997EC81807D4A2585129DA7F1C5BD5B, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:32,685 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520572448_0001_1_00_000000_0
2016-05-17 21:29:32,685 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A997EC81807D4A2585129DA7F1C5BD5B, mem on close (mb), free: 181, total: 356, max: 672
2016-05-17 21:29:32,686 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:32,686 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:32,686 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:32,686 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:32,690 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:32 Completed running task attempt: attempt_1463520572448_0001_1_00_000000_0
2016-05-17 21:29:32,693 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520572448_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:32,693 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520572448_0001_1_00, eventCount=1
2016-05-17 21:29:32,694 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:32,694 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A997EC81807D4A2585129DA7F1C5BD5B, taskAttemptId=attempt_1463520572448_0001_1_00_000000_0, startTime=1463520572635, finishTime=1463520572694, timeTaken=59, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373293056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520572683, Process_Duration=2, Process_End_Time=1463520572685, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:32,694 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520572448_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:32,695 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520572448_0001_1_00_000000_0
2016-05-17 21:29:32,695 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373293056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520572683, Process_Duration=2, Process_End_Time=1463520572685, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:32,695 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:32,695 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:TASK_FINISHED]: vertexName=A997EC81807D4A2585129DA7F1C5BD5B, taskId=task_1463520572448_0001_1_00_000000, startTime=1463520572635, finishTime=1463520572695, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1463520572448_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373293056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520572683, Process_Duration=2, Process_End_Time=1463520572685, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:32,696 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520572448_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:32,695 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:32,695 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:32,696 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:32,696 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B] : 1
2016-05-17 21:29:32,696 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B]
2016-05-17 21:29:32,696 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520572448_0001_00_000001
2016-05-17 21:29:32,698 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:32,698 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:VERTEX_FINISHED]: vertexName=A997EC81807D4A2585129DA7F1C5BD5B, vertexId=vertex_1463520572448_0001_1_00, initRequestedTime=1463520572610, initedTime=1463520572624, startRequestedTime=1463520572624, startedTime=1463520572624, finishTime=1463520572696, timeTaken=72, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373293056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520572683, Process_Duration=2, Process_End_Time=1463520572685, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520572635, firstTasksToStart=[ task_1463520572448_0001_1_00_000000 ], lastTaskFinishTime=1463520572694, lastTasksToFinish=[ task_1463520572448_0001_1_00_000000 ], minTaskDuration=59, maxTaskDuration=59, avgTaskDuration=59.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520572448_0001_1_00_000000 ], longestDurationTasks=[ task_1463520572448_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:32,698 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:32,698 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572448_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:32,698 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE] attempt: attempt_1463520572448_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:32,699 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F2CE52C638834D82A98E13067E2582FE
2016-05-17 21:29:32,699 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE]
2016-05-17 21:29:32,699 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520572448_0001_00_000001, stoppedTime=1463520572699, exitStatus=0
2016-05-17 21:29:32,699 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:32,699 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572448_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:32,699 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:32,700 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:TASK_STARTED]: vertexName=F2CE52C638834D82A98E13067E2582FE, taskId=task_1463520572448_0001_1_01_000000, scheduledTime=1463520572699, launchTime=1463520572699
2016-05-17 21:29:32,700 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520572448_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:32,700 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:32,700 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520572448_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:32,700 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE]
2016-05-17 21:29:32,700 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572448_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:32,701 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [42A5BEB476A74110AD5A51062150BABF/FF91F6BDB6E24CC4A39FF5C91AE2D593] every/(1/1) ...lyplatform/-1_VALUES_SWAP, VertexName: F2CE52C638834D82A98E13067E2582FE, VertexParallelism: 1, TaskAttemptID:attempt_1463520572448_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A997EC81807D4A2585129DA7F1C5BD5B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E9E72B71C87E478AA80F66EBA3620D0F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:32,701 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520572448_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:32,701 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:32,701 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520572448_0001_00_000002
2016-05-17 21:29:32,702 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572448_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:32,703 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520572448_0001_00_000002
2016-05-17 21:29:32,703 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:32,704 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572448_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:32,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:32,704 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520572448_0001_00_000002, launchTime=1463520572704
2016-05-17 21:29:32,705 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520572448_0001_00_000002
2016-05-17 21:29:32,706 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520572448_0001_1_01_000000_0] to container: [container_1463520572448_0001_00_000002]
2016-05-17 21:29:32,706 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572448_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:32,706 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520572448_0001_1_01_000000_0] started. Is using containerId: [container_1463520572448_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:32,706 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:32,706 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F2CE52C638834D82A98E13067E2582FE, taskAttemptId=attempt_1463520572448_0001_1_01_000000_0, startTime=1463520572706, containerId=container_1463520572448_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520572448_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:32,707 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520572448_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:32,707 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520572448_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:32,707 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520572448_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:32,707 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520572448_0001_00_000001, Exiting
2016-05-17 21:29:32,707 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520572448_0001_00_000001
2016-05-17 21:29:32,707 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520572448_0001_00_000001
2016-05-17 21:29:32,706 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520572448_0001_00_000002 given task: attempt_1463520572448_0001_1_01_000000_0
2016-05-17 21:29:32,707 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520572448_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:32,708 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520572448_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520572448_0001_1_01_000000_0
2016-05-17 21:29:32,708 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520572448_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:32 Starting to run new task attempt: attempt_1463520572448_0001_1_01_000000_0
2016-05-17 21:29:32,709 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520572448_0001_1_01_000000_0
2016-05-17 21:29:32,710 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:32,709 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520572448_0001_00_000001 completed successfully
2016-05-17 21:29:32,710 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:32,710 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572448_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:32,710 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:32,710 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:32,711 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:32,711 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:32,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [42A5BEB476A74110AD5A51062150BABF/FF91F6BDB6E24CC4A39FF5C91AE2D593] every/(1/1) ...lyplatform/-1_VALUES_SWAP, VertexName: F2CE52C638834D82A98E13067E2582FE, VertexParallelism: 1, TaskAttemptID:attempt_1463520572448_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A997EC81807D4A2585129DA7F1C5BD5B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E9E72B71C87E478AA80F66EBA3620D0F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:32,711 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:32,712 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520572448_0001_1_01_000000_0
2016-05-17 21:29:32,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:32,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:32,712 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520572448_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:32,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:32,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A997EC81807D4A2585129DA7F1C5BD5B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:32,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:32,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A997EC81807D4A2585129DA7F1C5BD5B
2016-05-17 21:29:32,715 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:32,715 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:32,715 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A997EC81807D4A2585129DA7F1C5BD5B
2016-05-17 21:29:32,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A997EC81807D4A2585129DA7F1C5BD5B
2016-05-17 21:29:32,716 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:32,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E9E72B71C87E478AA80F66EBA3620D0F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:32,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:32,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E9E72B71C87E478AA80F66EBA3620D0F
2016-05-17 21:29:32,718 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:32,718 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: E9E72B71C87E478AA80F66EBA3620D0F
2016-05-17 21:29:32,726 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:32,726 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E9E72B71C87E478AA80F66EBA3620D0F
2016-05-17 21:29:32,727 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F2CE52C638834D82A98E13067E2582FE, ordinal: 1
2016-05-17 21:29:32,727 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F2CE52C638834D82A98E13067E2582FE, mem on start (mb), free: 174, total: 356, max: 672
2016-05-17 21:29:32,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:32,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:32,728 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:32,728 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:32,728 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:32,728 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:32,729 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:32,729 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:32,729 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:32,729 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:32,730 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A997EC81807D4A2585129DA7F1C5BD5B, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:32,730 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E9E72B71C87E478AA80F66EBA3620D0F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:32,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:32,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A997EC81807D4A2585129DA7F1C5BD5B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:32,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A997EC81807D4A2585129DA7F1C5BD5B
2016-05-17 21:29:32,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:32,731 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:32,732 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:32,732 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:32,732 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A997EC81807D4A2585129DA7F1C5BD5B with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:32,732 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:32,732 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:32,732 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:32,734 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A997EC81807D4A2585129DA7F1C5BD5B
2016-05-17 21:29:32,734 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:32,734 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520572448_0001_1_01_000000_0
2016-05-17 21:29:32,737 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520572448_0001_1_00_000000_0_10087, runDuration: 47457, ]
2016-05-17 21:29:32,738 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520572448_0001_1_00_000000_0_10087] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520572738, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:32,739 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A997EC81807D4A2585129DA7F1C5BD5B
2016-05-17 21:29:32,739 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:32,739 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:32,739 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [A997EC81807D4A2585129DA7F1C5BD5B] #15 in 1ms
2016-05-17 21:29:32,740 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 66E1CDF71F16435FBB7ADC00BE848245
2016-05-17 21:29:32,740 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A997EC81807D4A2585129DA7F1C5BD5B] #15, status:false, isInterrupted:false
2016-05-17 21:29:32,741 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:32,741 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_SWAP"], id: E9E72B71C87E478AA80F66EBA3620D0F
2016-05-17 21:29:32,741 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_SWAP"]
2016-05-17 21:29:32,741 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 66E1CDF71F16435FBB7ADC00BE848245, for 1 inputs
2016-05-17 21:29:32,742 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:32,742 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:32,742 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:32,742 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:32,742 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:32,743 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A997EC81807D4A2585129DA7F1C5BD5B
2016-05-17 21:29:32,743 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:32,743 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F2CE52C638834D82A98E13067E2582FE, all 1 inputs ready in: 00:00:00.002
2016-05-17 21:29:32,744 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:32,745 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@1031fb8e; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@417a2ff0
2016-05-17 21:29:32,748 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:32,751 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:32,755 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:32,755 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:32,755 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520572448_0001_1_01_000000_0
2016-05-17 21:29:32,756 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520572448_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:32,756 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520572448_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:32,759 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373293056, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520572743, Process_Duration=13, Process_End_Time=1463520572756, Read_Duration=2, Tuples_Read=1]]
2016-05-17 21:29:32,759 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:32,759 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F2CE52C638834D82A98E13067E2582FE, taskAttemptId=attempt_1463520572448_0001_1_01_000000_0, startTime=1463520572706, finishTime=1463520572759, timeTaken=53, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373293056, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520572743, Process_Duration=13, Process_End_Time=1463520572756, Read_Duration=2, Tuples_Read=1
2016-05-17 21:29:32,760 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520572448_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:32,760 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520572448_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:32,760 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:TASK_FINISHED]: vertexName=F2CE52C638834D82A98E13067E2582FE, taskId=task_1463520572448_0001_1_01_000000, startTime=1463520572706, finishTime=1463520572760, timeTaken=54, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:32,760 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:32,759 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F2CE52C638834D82A98E13067E2582FE, mem on close (mb), free: 168, total: 356, max: 672
2016-05-17 21:29:32,760 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520572448_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:32,761 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE] : 1
2016-05-17 21:29:32,761 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE] because task failed: task_1463520572448_0001_1_01_000000
2016-05-17 21:29:32,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=F2CE52C638834D82A98E13067E2582FE, index=1
2016-05-17 21:29:32,761 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:32,762 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A997EC81807D4A2585129DA7F1C5BD5B
2016-05-17 21:29:32,761 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:32,762 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:32,762 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:32,762 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE]
2016-05-17 21:29:32,763 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=E9E72B71C87E478AA80F66EBA3620D0F, vertexId=vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE]
2016-05-17 21:29:32,764 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:32,764 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:32,764 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:32,764 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:VERTEX_FINISHED]: vertexName=F2CE52C638834D82A98E13067E2582FE, vertexId=vertex_1463520572448_0001_1_01, initRequestedTime=1463520572611, initedTime=1463520572618, startRequestedTime=1463520572627, startedTime=1463520572627, finishTime=1463520572764, timeTaken=137, status=FAILED, diagnostics=Task failed, taskId=task_1463520572448_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520572706, firstTasksToStart=[ task_1463520572448_0001_1_01_000000 ], lastTaskFinishTime=1463520572759, lastTasksToFinish=[ task_1463520572448_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:32,764 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:32,765 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:32,765 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520572448_0001_00_000002, stoppedTime=1463520572765, exitStatus=0
2016-05-17 21:29:32,765 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:32,765 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:32,765 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572448_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:32,766 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520572448_0001_1_01_000000_0 failed: true container: container_1463520572448_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:32,766 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:32,766 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:32,766 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520572448_0001_1
2016-05-17 21:29:32,766 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: E9E72B71C87E478AA80F66EBA3620D0F for vertex: vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE]
2016-05-17 21:29:32,767 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520572448_0001_1_00 [A997EC81807D4A2585129DA7F1C5BD5B]
2016-05-17 21:29:32,768 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520572448_0001_1, queueSize=0
2016-05-17 21:29:32,768 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:32,768 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572448_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520572448_0001_1, startTime=1463520572609, finishTime=1463520572767, timeTaken=158, status=FAILED, diagnostics=Vertex failed, vertexName=F2CE52C638834D82A98E13067E2582FE, vertexId=vertex_1463520572448_0001_1_01, diagnostics=[Task failed, taskId=task_1463520572448_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520572448_0001_1_01 [F2CE52C638834D82A98E13067E2582FE] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373293056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520572683, Process_Duration=2, Process_End_Time=1463520572685, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:32,768 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:32,768 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=F2CE52C638834D82A98E13067E2582FE, sourceVertex=A997EC81807D4A2585129DA7F1C5BD5B
2016-05-17 21:29:32,770 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520572448_0001_1 finished with state: FAILED
2016-05-17 21:29:32,770 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520572448_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:32,770 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520572448_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:32 Completed Dag: dag_1463520572448_0001_1
2016-05-17 21:29:32,770 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520572448_0001_1, dagState=FAILED
2016-05-17 21:29:32,770 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520572448_0001_1_post
2016-05-17 21:29:32,770 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:32,771 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:32,771 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572448_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:32,770 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=F2CE52C638834D82A98E13067E2582FE, sourceVertex=E9E72B71C87E478AA80F66EBA3620D0F
2016-05-17 21:29:32,771 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520572448_0001_00_000002
2016-05-17 21:29:32,771 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520572448_0001_00_000002
2016-05-17 21:29:32,771 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520572448_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:32,771 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520572448_0001_00_000002 completed but with errors
2016-05-17 21:29:32,772 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520572448_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:32,772 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572448_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:32,772 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520572448_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:32,783 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/-1_VALUES_SWAP, applicationId=application_1463520572448_0001
2016-05-17 21:29:32,784 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:32,784 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:32,784 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:32,785 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:32,785 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:32,785 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:32,785 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:32,785 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:32,786 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:32,786 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:32,786 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:32,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:32,787 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:32,791 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/42A5BEB476A74110AD5A51062150BABF/FF91F6BDB6E24CC4A39FF5C91AE2D593/.tez/application_1463520572448_0001
2016-05-17 21:29:32,791 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:32,791 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...lyplatform/-1_VALUES_SWAP
2016-05-17 21:29:32,792 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:32,792 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_SWAP/_temporary
2016-05-17 21:29:32,792 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.382
2016-05-17 21:29:32,793 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_SWAP
2016-05-17 21:29:32,819 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:32,819 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:32,868 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 21:29:32,879 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-05-17 21:29:32,880 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:32,880 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:32,880 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:32,896 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:32,902 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_-1
2016-05-17 21:29:32,944 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:32,944 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:32,944 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_-1"]
2016-05-17 21:29:32,945 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:32,945 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:32,945 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:32,947 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...semblyplatform/-1_ARGS_-1
2016-05-17 21:29:32,948 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:32,966 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:32,971 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:32,972 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:32,973 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:32,982 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/4DA5CA6820684D4A9288F0E04FEAC5DB/E58EDF08697547BD8D27394D93073D64/.tez/application_1463520572972_0001 doesn't exist and is created
2016-05-17 21:29:32,997 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:32,997 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:33,000 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:33,001 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:33,001 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/4DA5CA6820684D4A9288F0E04FEAC5DB/E58EDF08697547BD8D27394D93073D64/.tez/application_1463520572972_0001
2016-05-17 21:29:33,005 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520572972_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:33,020 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:33,021 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:33,023 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:33,023 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:33,024 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/4DA5CA6820684D4A9288F0E04FEAC5DB/E58EDF08697547BD8D27394D93073D64/.tez/application_1463520572972_0001/localmode-log-dir
2016-05-17 21:29:33,024 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/4DA5CA6820684D4A9288F0E04FEAC5DB/E58EDF08697547BD8D27394D93073D64/.tez/application_1463520572972_0001/localmode-log-dir/history.txt.appattempt_1463520572972_0001_000000, maxErrors=10
2016-05-17 21:29:33,024 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:33,025 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520572972_0001_000000, appSubmitTime=1463520573004, launchTime=1463520573004
2016-05-17 21:29:33,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:33,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:33,048 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:33,049 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:33,052 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:58278
2016-05-17 21:29:33,054 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:33,056 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520572972_0001_000000, startTime=1463520573056
2016-05-17 21:29:33,056 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:33,057 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:33,101 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:33,101 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:33,101 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...semblyplatform/-1_ARGS_-1, applicationId=application_1463520572972_0001, dagName=[4DA5CA6820684D4A9288F0E04FEAC5DB/E58EDF08697547BD8D27394D93073D64] every/(1/1) ...semblyplatform/-1_ARGS_-1
2016-05-17 21:29:33,125 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [4DA5CA6820684D4A9288F0E04FEAC5DB/E58EDF08697547BD8D27394D93073D64] every/(1/1) ...semblyplatform/-1_ARGS_-1
2016-05-17 21:29:33,127 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520572972_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/4DA5CA6820684D4A9288F0E04FEAC5DB/E58EDF08697547BD8D27394D93073D64/.tez/application_1463520572972_0001/localmode-log-dir/dag_1463520572972_0001_1.dot
2016-05-17 21:29:33,127 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/4DA5CA6820684D4A9288F0E04FEAC5DB/E58EDF08697547BD8D27394D93073D64/.tez/application_1463520572972_0001/localmode-log-dir/dag_1463520572972_0001_1-tez-dag.pb.txt
2016-05-17 21:29:33,141 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520572972_0001_1
2016-05-17 21:29:33,141 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:33,142 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [4DA5CA6820684D4A9288F0E04FEAC5DB/E58EDF08697547BD8D27394D93073D64] every/(1/1) ...semblyplatform/-1_ARGS_-1
2016-05-17 21:29:33 Running Dag: dag_1463520572972_0001_1
2016-05-17 21:29:33,153 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:33,153 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520572972_0001_1, submitTime=1463520573125
2016-05-17 21:29:33,154 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A2EFF91A223D4BFDB4C09EEDD42E0BCB
2016-05-17 21:29:33,154 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:33,155 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:33,156 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520572972_0001_1, initTime=1463520573153
2016-05-17 21:29:33,156 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520572972_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:33,156 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:DAG_STARTED]: dagID=dag_1463520572972_0001_1, startTime=1463520573156
2016-05-17 21:29:33,156 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:33,156 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:33,158 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520572972_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:33,159 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CBCE134DB370407198CDB1B39F327752 : {99792FDFFE6E4FD79C9A009400CCB2BB={InputName=99792FDFFE6E4FD79C9A009400CCB2BB}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:33,159 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 99792FDFFE6E4FD79C9A009400CCB2BB, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:33,159 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752]
2016-05-17 21:29:33,159 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...semblyplatform/-1_ARGS_-1, applicationId=application_1463520572972_0001, dagName=[4DA5CA6820684D4A9288F0E04FEAC5DB/E58EDF08697547BD8D27394D93073D64] every/(1/1) ...semblyplatform/-1_ARGS_-1
2016-05-17 21:29:33,159 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752]
2016-05-17 21:29:33,160 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752]
2016-05-17 21:29:33,160 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752]. Starting root input initializers: 1
2016-05-17 21:29:33,160 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:33,161 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB]
2016-05-17 21:29:33,161 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 99792FDFFE6E4FD79C9A009400CCB2BB on vertex vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752]
2016-05-17 21:29:33,164 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:33,164 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB]
2016-05-17 21:29:33,164 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB]
2016-05-17 21:29:33,165 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB]
2016-05-17 21:29:33,165 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=E2A903136C7547B49F991B12CFA7B14E, vertexId=vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:33,166 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=E2A903136C7547B49F991B12CFA7B14E, vertexId=vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB]
2016-05-17 21:29:33,167 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:33,167 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 99792FDFFE6E4FD79C9A009400CCB2BB asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:33,168 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=E2A903136C7547B49F991B12CFA7B14E, vertexName=A2EFF91A223D4BFDB4C09EEDD42E0BCB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:33,168 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:33,168 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:33,169 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=E2A903136C7547B49F991B12CFA7B14E, vertexId=vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB]
2016-05-17 21:29:33,170 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:33,170 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:33,170 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A2EFF91A223D4BFDB4C09EEDD42E0BCB, vertexId=vertex_1463520572972_0001_1_01, initRequestedTime=1463520573161, initedTime=1463520573170, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:33,171 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:33,171 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 99792FDFFE6E4FD79C9A009400CCB2BB on vertex vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752]
2016-05-17 21:29:33,171 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:33,171 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {99792FDFFE6E4FD79C9A009400CCB2BB=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:33,171 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752] parallelism set to 1
2016-05-17 21:29:33,172 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:33,172 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520572972_0001_1_00, eventCount=1
2016-05-17 21:29:33,172 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:33,172 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CBCE134DB370407198CDB1B39F327752, vertexId=vertex_1463520572972_0001_1_00, initRequestedTime=1463520573159, initedTime=1463520573172, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:33,173 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752] with distanceFromRoot: 0
2016-05-17 21:29:33,173 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:33,173 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CBCE134DB370407198CDB1B39F327752
2016-05-17 21:29:33,173 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752]
2016-05-17 21:29:33,173 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:33,173 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520572972_0001_1_00, startRequestedTime=1463520573173, startedTime=1463520573173
2016-05-17 21:29:33,174 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:33,174 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:33,174 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:TASK_STARTED]: vertexName=CBCE134DB370407198CDB1B39F327752, taskId=task_1463520572972_0001_1_00_000000, scheduledTime=1463520573174, launchTime=1463520573174
2016-05-17 21:29:33,174 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520572972_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:33,174 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520572972_0001_1_00 for vertex: vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB] numStartedSources: 1 numSources: 1
2016-05-17 21:29:33,175 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB] with distanceFromRoot: 1
2016-05-17 21:29:33,175 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752]
2016-05-17 21:29:33,175 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CBCE134DB370407198CDB1B39F327752 in vertex: A2EFF91A223D4BFDB4C09EEDD42E0BCB
2016-05-17 21:29:33,175 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A2EFF91A223D4BFDB4C09EEDD42E0BCB with 1 source tasks and 1 pending tasks
2016-05-17 21:29:33,175 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:33,176 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520572972_0001_1_01, startRequestedTime=1463520573175, startedTime=1463520573175
2016-05-17 21:29:33,176 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:33,176 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [4DA5CA6820684D4A9288F0E04FEAC5DB/E58EDF08697547BD8D27394D93073D64] every/(1/1) ...semblyplatform/-1_ARGS_-1, VertexName: CBCE134DB370407198CDB1B39F327752, VertexParallelism: 1, TaskAttemptID:attempt_1463520572972_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=99792FDFFE6E4FD79C9A009400CCB2BB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A2EFF91A223D4BFDB4C09EEDD42E0BCB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:33,176 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520572972_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:33,176 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:33,177 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:33,177 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:33,178 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:33,178 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520572972_0001_00_000001
2016-05-17 21:29:33,178 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572972_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:33,180 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520572972_0001, with dag id: dag_1463520572972_0001_1
2016-05-17 21:29:33,181 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520572972_0001_00_000001
2016-05-17 21:29:33,181 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:33,182 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572972_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:33,182 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:33,182 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520572972_0001_00_000001, launchTime=1463520573182
2016-05-17 21:29:33,183 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520572972_0001_00_000001
2016-05-17 21:29:33,184 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520572972_0001_1_00_000000_0] to container: [container_1463520572972_0001_00_000001]
2016-05-17 21:29:33,184 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572972_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:33,185 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520572972_0001_00_000001 given task: attempt_1463520572972_0001_1_00_000000_0
2016-05-17 21:29:33,185 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520572972_0001_1_00_000000_0] started. Is using containerId: [container_1463520572972_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:33,185 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CBCE134DB370407198CDB1B39F327752, taskAttemptId=attempt_1463520572972_0001_1_00_000000_0, startTime=1463520573185, containerId=container_1463520572972_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520572972_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:33,185 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520572972_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:33,185 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520572972_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:33,186 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520572972_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520572972_0001_1_00_000000_0
2016-05-17 21:29:33,186 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:33 Starting to run new task attempt: attempt_1463520572972_0001_1_00_000000_0
2016-05-17 21:29:33,186 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520572972_0001_1_00_000000_0
2016-05-17 21:29:33,186 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:33,186 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:33,186 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:33,186 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:33,186 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:33,187 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:33,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4DA5CA6820684D4A9288F0E04FEAC5DB/E58EDF08697547BD8D27394D93073D64] every/(1/1) ...semblyplatform/-1_ARGS_-1, VertexName: CBCE134DB370407198CDB1B39F327752, VertexParallelism: 1, TaskAttemptID:attempt_1463520572972_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=99792FDFFE6E4FD79C9A009400CCB2BB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A2EFF91A223D4BFDB4C09EEDD42E0BCB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:33,187 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:33,188 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520572972_0001_1_00_000000_0
2016-05-17 21:29:33,188 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:33,188 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:33,189 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520572972_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:33,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=99792FDFFE6E4FD79C9A009400CCB2BB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:33,190 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:33,190 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 99792FDFFE6E4FD79C9A009400CCB2BB
2016-05-17 21:29:33,190 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 99792FDFFE6E4FD79C9A009400CCB2BB
2016-05-17 21:29:33,190 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:33,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A2EFF91A223D4BFDB4C09EEDD42E0BCB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:33,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:33,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A2EFF91A223D4BFDB4C09EEDD42E0BCB
2016-05-17 21:29:33,194 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:33,194 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 99792FDFFE6E4FD79C9A009400CCB2BB
2016-05-17 21:29:33,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 99792FDFFE6E4FD79C9A009400CCB2BB
2016-05-17 21:29:33,193 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:33,196 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:33,196 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A2EFF91A223D4BFDB4C09EEDD42E0BCB
2016-05-17 21:29:33,197 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A2EFF91A223D4BFDB4C09EEDD42E0BCB
2016-05-17 21:29:33,200 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CBCE134DB370407198CDB1B39F327752, ordinal: 0
2016-05-17 21:29:33,200 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CBCE134DB370407198CDB1B39F327752, mem on start (mb), free: 299, total: 359, max: 672
2016-05-17 21:29:33,200 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:33,200 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:33,200 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:33,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:33,201 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:33,201 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:33,201 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:33,202 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:33,202 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:33,202 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:33,202 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 99792FDFFE6E4FD79C9A009400CCB2BB, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:33,203 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A2EFF91A223D4BFDB4C09EEDD42E0BCB, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:33,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:33,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 99792FDFFE6E4FD79C9A009400CCB2BB
2016-05-17 21:29:33,203 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:33,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 99792FDFFE6E4FD79C9A009400CCB2BB
2016-05-17 21:29:33,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 99792FDFFE6E4FD79C9A009400CCB2BB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:33,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:33,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:33,205 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:33,205 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520572972_0001_1_00_000000_0
2016-05-17 21:29:33,205 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:33,206 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1652f498
2016-05-17 21:29:33,207 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1652f498
2016-05-17 21:29:33,207 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:33,207 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:33,208 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:33,209 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 99792FDFFE6E4FD79C9A009400CCB2BB
2016-05-17 21:29:33,209 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 55AC6B6E7FD742A7A6E182D283A48FA0
2016-05-17 21:29:33,209 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 55AC6B6E7FD742A7A6E182D283A48FA0
2016-05-17 21:29:33,210 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:33,212 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:33,213 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@281b2cc1; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@10174779; comparator=cascading.tuple.hadoop.util.TupleComparator@6855612f; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:33,213 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:33,213 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:33,214 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:33,227 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:33,227 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:33,227 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:33,227 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:33,228 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:33,228 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:33,228 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:33,229 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CBCE134DB370407198CDB1B39F327752, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:29:33,231 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520572972_0001_1_00_000000_0
2016-05-17 21:29:33,231 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CBCE134DB370407198CDB1B39F327752, mem on close (mb), free: 197, total: 359, max: 672
2016-05-17 21:29:33,231 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:33,231 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:33,232 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:33,232 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:33,235 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:33 Completed running task attempt: attempt_1463520572972_0001_1_00_000000_0
2016-05-17 21:29:33,238 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520572972_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:33,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520573229, Process_Duration=2, Process_End_Time=1463520573231, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:33,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520572972_0001_1_00, eventCount=1
2016-05-17 21:29:33,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CBCE134DB370407198CDB1B39F327752, taskAttemptId=attempt_1463520572972_0001_1_00_000000_0, startTime=1463520573185, finishTime=1463520573239, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520573229, Process_Duration=2, Process_End_Time=1463520573231, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:33,239 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520572972_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:33,239 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520572972_0001_1_00_000000_0
2016-05-17 21:29:33,239 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:33,240 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:33,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:TASK_FINISHED]: vertexName=CBCE134DB370407198CDB1B39F327752, taskId=task_1463520572972_0001_1_00_000000, startTime=1463520573185, finishTime=1463520573239, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1463520572972_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520573229, Process_Duration=2, Process_End_Time=1463520573231, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:33,240 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520572972_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:33,240 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752] : 1
2016-05-17 21:29:33,240 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752]
2016-05-17 21:29:33,240 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:33,241 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:33,241 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:33,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:33,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:VERTEX_FINISHED]: vertexName=CBCE134DB370407198CDB1B39F327752, vertexId=vertex_1463520572972_0001_1_00, initRequestedTime=1463520573159, initedTime=1463520573172, startRequestedTime=1463520573173, startedTime=1463520573173, finishTime=1463520573240, timeTaken=67, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520573229, Process_Duration=2, Process_End_Time=1463520573231, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520573185, firstTasksToStart=[ task_1463520572972_0001_1_00_000000 ], lastTaskFinishTime=1463520573239, lastTasksToFinish=[ task_1463520572972_0001_1_00_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520572972_0001_1_00_000000 ], longestDurationTasks=[ task_1463520572972_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:33,242 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:33,242 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572972_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:33,242 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB] attempt: attempt_1463520572972_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:33,242 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A2EFF91A223D4BFDB4C09EEDD42E0BCB
2016-05-17 21:29:33,242 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB]
2016-05-17 21:29:33,242 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520572972_0001_00_000001
2016-05-17 21:29:33,242 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:33,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520572972_0001_00_000001, stoppedTime=1463520573242, exitStatus=0
2016-05-17 21:29:33,243 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572972_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:33,243 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:33,243 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:TASK_STARTED]: vertexName=A2EFF91A223D4BFDB4C09EEDD42E0BCB, taskId=task_1463520572972_0001_1_01_000000, scheduledTime=1463520573243, launchTime=1463520573243
2016-05-17 21:29:33,243 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520572972_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:33,245 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520572972_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:33,245 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520572972_0001_00_000001, Exiting
2016-05-17 21:29:33,245 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520572972_0001_00_000001
2016-05-17 21:29:33,245 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520572972_0001_00_000001
2016-05-17 21:29:33,244 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520572972_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:33,246 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB]
2016-05-17 21:29:33,246 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [4DA5CA6820684D4A9288F0E04FEAC5DB/E58EDF08697547BD8D27394D93073D64] every/(1/1) ...semblyplatform/-1_ARGS_-1, VertexName: A2EFF91A223D4BFDB4C09EEDD42E0BCB, VertexParallelism: 1, TaskAttemptID:attempt_1463520572972_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CBCE134DB370407198CDB1B39F327752, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E2A903136C7547B49F991B12CFA7B14E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:33,246 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520572972_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:33,244 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:33,247 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:33,246 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520572972_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:33,246 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520572972_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:33,247 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520572972_0001_00_000001 completed successfully
2016-05-17 21:29:33,248 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520572972_0001_00_000002
2016-05-17 21:29:33,248 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572972_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:33,248 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572972_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:33,248 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572972_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:33,250 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520572972_0001_00_000002
2016-05-17 21:29:33,251 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:33,253 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572972_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:33,253 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:33,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520572972_0001_00_000002, launchTime=1463520573253
2016-05-17 21:29:33,254 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520572972_0001_00_000002
2016-05-17 21:29:33,254 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520572972_0001_1_01_000000_0] to container: [container_1463520572972_0001_00_000002]
2016-05-17 21:29:33,254 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572972_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:33,255 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520572972_0001_00_000002 given task: attempt_1463520572972_0001_1_01_000000_0
2016-05-17 21:29:33,255 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520572972_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520572972_0001_1_01_000000_0
2016-05-17 21:29:33,255 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520572972_0001_1_01_000000_0] started. Is using containerId: [container_1463520572972_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:33,255 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A2EFF91A223D4BFDB4C09EEDD42E0BCB, taskAttemptId=attempt_1463520572972_0001_1_01_000000_0, startTime=1463520573255, containerId=container_1463520572972_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520572972_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:33,255 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:33,255 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520572972_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:33,255 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520572972_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:33 Starting to run new task attempt: attempt_1463520572972_0001_1_01_000000_0
2016-05-17 21:29:33,256 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520572972_0001_1_01_000000_0
2016-05-17 21:29:33,256 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:33,257 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:33,257 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:33,257 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:33,257 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:33,257 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:33,258 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4DA5CA6820684D4A9288F0E04FEAC5DB/E58EDF08697547BD8D27394D93073D64] every/(1/1) ...semblyplatform/-1_ARGS_-1, VertexName: A2EFF91A223D4BFDB4C09EEDD42E0BCB, VertexParallelism: 1, TaskAttemptID:attempt_1463520572972_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CBCE134DB370407198CDB1B39F327752, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E2A903136C7547B49F991B12CFA7B14E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:33,258 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:33,258 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520572972_0001_1_01_000000_0
2016-05-17 21:29:33,258 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:33,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:33,259 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520572972_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:33,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CBCE134DB370407198CDB1B39F327752, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:33,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E2A903136C7547B49F991B12CFA7B14E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:33,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:33,262 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:33,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:33,262 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CBCE134DB370407198CDB1B39F327752
2016-05-17 21:29:33,264 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:33,264 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:33,264 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CBCE134DB370407198CDB1B39F327752
2016-05-17 21:29:33,264 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CBCE134DB370407198CDB1B39F327752
2016-05-17 21:29:33,265 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:33,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E2A903136C7547B49F991B12CFA7B14E
2016-05-17 21:29:33,271 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:33,271 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: E2A903136C7547B49F991B12CFA7B14E
2016-05-17 21:29:33,278 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A2EFF91A223D4BFDB4C09EEDD42E0BCB, ordinal: 1
2016-05-17 21:29:33,278 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A2EFF91A223D4BFDB4C09EEDD42E0BCB, mem on start (mb), free: 190, total: 359, max: 672
2016-05-17 21:29:33,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:33,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:33,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:33,285 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:33,285 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E2A903136C7547B49F991B12CFA7B14E
2016-05-17 21:29:33,285 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:33,285 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:33,285 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:33,285 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:33,285 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:33,286 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:33,286 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:33,286 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CBCE134DB370407198CDB1B39F327752, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:33,286 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E2A903136C7547B49F991B12CFA7B14E, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:33,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:33,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CBCE134DB370407198CDB1B39F327752
2016-05-17 21:29:33,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CBCE134DB370407198CDB1B39F327752 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:33,287 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:33,288 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:33,288 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CBCE134DB370407198CDB1B39F327752 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:33,288 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:33,288 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:33,288 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:33,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CBCE134DB370407198CDB1B39F327752
2016-05-17 21:29:33,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:33,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:33,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:33,290 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520572972_0001_1_01_000000_0
2016-05-17 21:29:33,290 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520572972_0001_1_00_000000_0_10093, runDuration: 42940, ]
2016-05-17 21:29:33,293 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:33,293 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520572972_0001_1_00_000000_0_10093] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520573293, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:33,294 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 55AC6B6E7FD742A7A6E182D283A48FA0
2016-05-17 21:29:33,294 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_-1"], id: E2A903136C7547B49F991B12CFA7B14E
2016-05-17 21:29:33,294 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_-1"]
2016-05-17 21:29:33,294 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CBCE134DB370407198CDB1B39F327752
2016-05-17 21:29:33,296 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:33,295 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 55AC6B6E7FD742A7A6E182D283A48FA0, for 1 inputs
2016-05-17 21:29:33,296 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [CBCE134DB370407198CDB1B39F327752] #16 in 4ms
2016-05-17 21:29:33,296 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CBCE134DB370407198CDB1B39F327752] #16, status:false, isInterrupted:false
2016-05-17 21:29:33,297 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:33,298 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:33,298 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:33,299 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:33,299 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:33,299 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:33,299 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CBCE134DB370407198CDB1B39F327752
2016-05-17 21:29:33,299 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:33,299 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A2EFF91A223D4BFDB4C09EEDD42E0BCB, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:33,301 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:33,304 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@3b977351; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@5413e967
2016-05-17 21:29:33,307 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:33,307 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:33,307 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520572972_0001_1_01_000000_0
2016-05-17 21:29:33,308 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520572972_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:33,309 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520572972_0001_1_01_000000_0
2016-05-17 21:29:33,309 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A2EFF91A223D4BFDB4C09EEDD42E0BCB, mem on close (mb), free: 186, total: 359, max: 672
2016-05-17 21:29:33,309 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CBCE134DB370407198CDB1B39F327752
2016-05-17 21:29:33,309 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:33,312 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:33,312 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:33,312 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:33,312 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:33,312 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:33 Completed running task attempt: attempt_1463520572972_0001_1_01_000000_0
2016-05-17 21:29:33,313 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520572972_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:33,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520573299, Process_Duration=9, Process_End_Time=1463520573308, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:33,319 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:33,319 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:33,319 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:33,319 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520572972_0001_00_000002
2016-05-17 21:29:33,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A2EFF91A223D4BFDB4C09EEDD42E0BCB, taskAttemptId=attempt_1463520572972_0001_1_01_000000_0, startTime=1463520573255, finishTime=1463520573318, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520573299, Process_Duration=9, Process_End_Time=1463520573308, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:33,320 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520572972_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:33,320 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520572972_0001_1_01_000000_0
2016-05-17 21:29:33,321 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:TASK_FINISHED]: vertexName=A2EFF91A223D4BFDB4C09EEDD42E0BCB, taskId=task_1463520572972_0001_1_01_000000, startTime=1463520573255, finishTime=1463520573320, timeTaken=65, status=SUCCEEDED, successfulAttemptID=attempt_1463520572972_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520573299, Process_Duration=9, Process_End_Time=1463520573308, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:33,321 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:33,321 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520572972_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:33,321 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB] : 1
2016-05-17 21:29:33,321 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB]
2016-05-17 21:29:33,321 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:33,322 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:VERTEX_FINISHED]: vertexName=A2EFF91A223D4BFDB4C09EEDD42E0BCB, vertexId=vertex_1463520572972_0001_1_01, initRequestedTime=1463520573161, initedTime=1463520573170, startRequestedTime=1463520573175, startedTime=1463520573175, finishTime=1463520573321, timeTaken=146, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=376963072, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520573299, Process_Duration=9, Process_End_Time=1463520573308, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520573255, firstTasksToStart=[ task_1463520572972_0001_1_01_000000 ], lastTaskFinishTime=1463520573318, lastTasksToFinish=[ task_1463520572972_0001_1_01_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520572972_0001_1_01_000000 ], longestDurationTasks=[ task_1463520572972_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:33,323 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:33,323 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:33,323 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572972_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:33,323 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520572972_0001_00_000002, stoppedTime=1463520573323, exitStatus=0
2016-05-17 21:29:33,324 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572972_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:33,324 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:33,324 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:33,324 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520572972_0001_1
2016-05-17 21:29:33,324 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520572972_0001_1
2016-05-17 21:29:33,324 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520572972_0001_1_00 [CBCE134DB370407198CDB1B39F327752]
2016-05-17 21:29:33,324 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: E2A903136C7547B49F991B12CFA7B14E for vertex: vertex_1463520572972_0001_1_01 [A2EFF91A223D4BFDB4C09EEDD42E0BCB]
2016-05-17 21:29:33,332 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520572972_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:33,332 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520572972_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:33,333 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520572972_0001_00_000002, Exiting
2016-05-17 21:29:33,333 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520572972_0001_00_000002
2016-05-17 21:29:33,333 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520572972_0001_00_000002
2016-05-17 21:29:33,333 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520572972_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:33,333 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520572972_0001_00_000002 completed successfully
2016-05-17 21:29:33,333 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520572972_0001_1, queueSize=0
2016-05-17 21:29:33,334 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:33,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520572972_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520572972_0001_1, startTime=1463520573156, finishTime=1463520573330, timeTaken=174, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=345, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=753926144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041146528, Process_Duration=11, Process_End_Time=2927041146539, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:33,334 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520572972_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:33,334 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520572972_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:33,335 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520572972_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:33,335 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520572972_0001_00_000002
2016-05-17 21:29:33 Completed Dag: dag_1463520572972_0001_1
2016-05-17 21:29:33,338 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520572972_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:33,339 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520572972_0001_1_post
2016-05-17 21:29:33,341 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:33,341 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:33,341 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...semblyplatform/-1_ARGS_-1, applicationId=application_1463520572972_0001
2016-05-17 21:29:33,343 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:33,343 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:33,343 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:33,343 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:33,343 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:33,344 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:33,344 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:33,344 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:33,345 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:33,345 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:33,345 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:33,345 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:33,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:33,350 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/4DA5CA6820684D4A9288F0E04FEAC5DB/E58EDF08697547BD8D27394D93073D64/.tez/application_1463520572972_0001
2016-05-17 21:29:33,350 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_-1/_temporary
2016-05-17 21:29:33,350 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.406
2016-05-17 21:29:33,414 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_-1
2016-05-17 21:29:33,440 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:33,441 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:33,473 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-17 21:29:33,478 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.010
2016-05-17 21:29:33,478 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:33,479 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:33,479 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_'field'
2016-05-17 21:29:33,503 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:33,503 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:33,539 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 21:29:33,546 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-17 21:29:33,546 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:33,546 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:33,547 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:33,561 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:33,567 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_RESULTS
2016-05-17 21:29:33,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:33,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:33,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_RESULTS"]
2016-05-17 21:29:33,627 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:33,627 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:33,627 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:33,628 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...yplatform/-1_ARGS_RESULTS
2016-05-17 21:29:33,628 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:33,641 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:33,646 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:33,647 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:33,647 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:33,656 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79D0A1DE833A4971A84754D3F43F776E/F29B55279ED849A9A4515F5DA6660A49/.tez/application_1463520573646_0001 doesn't exist and is created
2016-05-17 21:29:33,663 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:33,663 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:33,670 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:33,670 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:33,671 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79D0A1DE833A4971A84754D3F43F776E/F29B55279ED849A9A4515F5DA6660A49/.tez/application_1463520573646_0001
2016-05-17 21:29:33,673 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520573646_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:33,683 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:33,683 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:33,686 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:33,686 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:33,687 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79D0A1DE833A4971A84754D3F43F776E/F29B55279ED849A9A4515F5DA6660A49/.tez/application_1463520573646_0001/localmode-log-dir
2016-05-17 21:29:33,687 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79D0A1DE833A4971A84754D3F43F776E/F29B55279ED849A9A4515F5DA6660A49/.tez/application_1463520573646_0001/localmode-log-dir/history.txt.appattempt_1463520573646_0001_000000, maxErrors=10
2016-05-17 21:29:33,687 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:33,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520573646_0001_000000, appSubmitTime=1463520573673, launchTime=1463520573673
2016-05-17 21:29:33,690 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:33,693 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:33,694 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:33,694 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:33,695 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:33527
2016-05-17 21:29:33,697 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:33,699 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520573646_0001_000000, startTime=1463520573699
2016-05-17 21:29:33,699 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:33,699 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:33,770 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:33,771 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:33,771 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_RESULTS, applicationId=application_1463520573646_0001, dagName=[79D0A1DE833A4971A84754D3F43F776E/F29B55279ED849A9A4515F5DA6660A49] every/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-05-17 21:29:33,785 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [79D0A1DE833A4971A84754D3F43F776E/F29B55279ED849A9A4515F5DA6660A49] every/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-05-17 21:29:33,787 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520573646_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79D0A1DE833A4971A84754D3F43F776E/F29B55279ED849A9A4515F5DA6660A49/.tez/application_1463520573646_0001/localmode-log-dir/dag_1463520573646_0001_1.dot
2016-05-17 21:29:33,787 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79D0A1DE833A4971A84754D3F43F776E/F29B55279ED849A9A4515F5DA6660A49/.tez/application_1463520573646_0001/localmode-log-dir/dag_1463520573646_0001_1-tez-dag.pb.txt
2016-05-17 21:29:33,796 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520573646_0001_1
2016-05-17 21:29:33,796 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:33,796 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [79D0A1DE833A4971A84754D3F43F776E/F29B55279ED849A9A4515F5DA6660A49] every/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-05-17 21:29:33 Running Dag: dag_1463520573646_0001_1
2016-05-17 21:29:33,807 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520573646_0001_1, submitTime=1463520573786
2016-05-17 21:29:33,807 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:33,808 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A1636AFCD7F34445A53E917E493DD1AF
2016-05-17 21:29:33,808 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:33,809 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520573646_0001_1, initTime=1463520573807
2016-05-17 21:29:33,810 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520573646_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:33,810 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:DAG_STARTED]: dagID=dag_1463520573646_0001_1, startTime=1463520573810
2016-05-17 21:29:33,810 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:33,810 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520573646_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:33,810 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F217EAD43FDD440299B9869285565B9A : {F6A557A45A4241DF9ACBB497FC9DFF35={InputName=F6A557A45A4241DF9ACBB497FC9DFF35}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:33,810 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F6A557A45A4241DF9ACBB497FC9DFF35, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:33,811 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A]
2016-05-17 21:29:33,811 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A]
2016-05-17 21:29:33,811 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A]
2016-05-17 21:29:33,811 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A]. Starting root input initializers: 1
2016-05-17 21:29:33,812 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:33,812 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:33,813 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:33,813 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF]
2016-05-17 21:29:33,813 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F6A557A45A4241DF9ACBB497FC9DFF35 on vertex vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A]
2016-05-17 21:29:33,815 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:33,816 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F6A557A45A4241DF9ACBB497FC9DFF35 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:33,821 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:33,821 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:33,819 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:33,822 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF]
2016-05-17 21:29:33,822 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF]
2016-05-17 21:29:33,822 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF]
2016-05-17 21:29:33,822 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=42C3DD000FBA4CA2B9755EA0FAE999F4, vertexId=vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:33,822 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_RESULTS, applicationId=application_1463520573646_0001, dagName=[79D0A1DE833A4971A84754D3F43F776E/F29B55279ED849A9A4515F5DA6660A49] every/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-05-17 21:29:33,823 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=42C3DD000FBA4CA2B9755EA0FAE999F4, vertexId=vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF]
2016-05-17 21:29:33,827 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=42C3DD000FBA4CA2B9755EA0FAE999F4, vertexName=A1636AFCD7F34445A53E917E493DD1AF, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:33,828 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=42C3DD000FBA4CA2B9755EA0FAE999F4, vertexId=vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF]
2016-05-17 21:29:33,825 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:33,829 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F6A557A45A4241DF9ACBB497FC9DFF35 on vertex vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A]
2016-05-17 21:29:33,830 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A1636AFCD7F34445A53E917E493DD1AF, vertexId=vertex_1463520573646_0001_1_01, initRequestedTime=1463520573813, initedTime=1463520573830, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:33,830 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:33,830 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:33,831 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:33,831 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F6A557A45A4241DF9ACBB497FC9DFF35=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:33,832 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A] parallelism set to 1
2016-05-17 21:29:33,832 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:33,832 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520573646_0001_1_00, eventCount=1
2016-05-17 21:29:33,834 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:33,835 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F217EAD43FDD440299B9869285565B9A, vertexId=vertex_1463520573646_0001_1_00, initRequestedTime=1463520573810, initedTime=1463520573834, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:33,835 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A] with distanceFromRoot: 0
2016-05-17 21:29:33,835 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:33,835 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F217EAD43FDD440299B9869285565B9A
2016-05-17 21:29:33,835 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A]
2016-05-17 21:29:33,835 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:33,836 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520573646_0001_1_00, startRequestedTime=1463520573835, startedTime=1463520573835
2016-05-17 21:29:33,836 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:33,836 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:33,836 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:TASK_STARTED]: vertexName=F217EAD43FDD440299B9869285565B9A, taskId=task_1463520573646_0001_1_00_000000, scheduledTime=1463520573836, launchTime=1463520573836
2016-05-17 21:29:33,836 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520573646_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:33,837 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520573646_0001_1_00 for vertex: vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF] numStartedSources: 1 numSources: 1
2016-05-17 21:29:33,837 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF] with distanceFromRoot: 1
2016-05-17 21:29:33,837 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A]
2016-05-17 21:29:33,837 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F217EAD43FDD440299B9869285565B9A in vertex: A1636AFCD7F34445A53E917E493DD1AF
2016-05-17 21:29:33,837 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A1636AFCD7F34445A53E917E493DD1AF with 1 source tasks and 1 pending tasks
2016-05-17 21:29:33,838 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:33,838 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520573646_0001_1_01, startRequestedTime=1463520573837, startedTime=1463520573837
2016-05-17 21:29:33,838 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:33,838 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [79D0A1DE833A4971A84754D3F43F776E/F29B55279ED849A9A4515F5DA6660A49] every/(1/1) ...yplatform/-1_ARGS_RESULTS, VertexName: F217EAD43FDD440299B9869285565B9A, VertexParallelism: 1, TaskAttemptID:attempt_1463520573646_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F6A557A45A4241DF9ACBB497FC9DFF35, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A1636AFCD7F34445A53E917E493DD1AF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:33,838 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520573646_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:33,838 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:33,839 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:33,839 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:33,840 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:33,840 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520573646_0001_00_000001
2016-05-17 21:29:33,840 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520573646_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:33,842 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520573646_0001_00_000001
2016-05-17 21:29:33,842 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:33,843 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520573646_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:33,843 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:33,843 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520573646_0001_00_000001, launchTime=1463520573843
2016-05-17 21:29:33,845 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520573646_0001_00_000001
2016-05-17 21:29:33,845 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520573646_0001_1_00_000000_0] to container: [container_1463520573646_0001_00_000001]
2016-05-17 21:29:33,845 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520573646_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:33,846 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520573646_0001_1_00_000000_0] started. Is using containerId: [container_1463520573646_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:33,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F217EAD43FDD440299B9869285565B9A, taskAttemptId=attempt_1463520573646_0001_1_00_000000_0, startTime=1463520573846, containerId=container_1463520573646_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520573646_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:33,846 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520573646_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:33,847 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520573646_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:33,847 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:33,847 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520573646_0001_00_000001 given task: attempt_1463520573646_0001_1_00_000000_0
2016-05-17 21:29:33,847 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520573646_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520573646_0001_1_00_000000_0
2016-05-17 21:29:33 Starting to run new task attempt: attempt_1463520573646_0001_1_00_000000_0
2016-05-17 21:29:33,848 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520573646_0001_1_00_000000_0
2016-05-17 21:29:33,848 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:33,848 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:33,848 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:33,848 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:33,848 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:33,849 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:33,849 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [79D0A1DE833A4971A84754D3F43F776E/F29B55279ED849A9A4515F5DA6660A49] every/(1/1) ...yplatform/-1_ARGS_RESULTS, VertexName: F217EAD43FDD440299B9869285565B9A, VertexParallelism: 1, TaskAttemptID:attempt_1463520573646_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F6A557A45A4241DF9ACBB497FC9DFF35, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A1636AFCD7F34445A53E917E493DD1AF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:33,849 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:33,850 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520573646_0001_1_00_000000_0
2016-05-17 21:29:33,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:33,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:33,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F6A557A45A4241DF9ACBB497FC9DFF35, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:33,852 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:33,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F6A557A45A4241DF9ACBB497FC9DFF35
2016-05-17 21:29:33,853 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F6A557A45A4241DF9ACBB497FC9DFF35
2016-05-17 21:29:33,854 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520573646_0001, with dag id: dag_1463520573646_0001_1
2016-05-17 21:29:33,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:33,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A1636AFCD7F34445A53E917E493DD1AF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:33,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:33,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A1636AFCD7F34445A53E917E493DD1AF
2016-05-17 21:29:33,856 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:33,856 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A1636AFCD7F34445A53E917E493DD1AF
2016-05-17 21:29:33,857 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:33,859 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520573646_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:33,860 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:33,860 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F6A557A45A4241DF9ACBB497FC9DFF35
2016-05-17 21:29:33,861 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F6A557A45A4241DF9ACBB497FC9DFF35
2016-05-17 21:29:33,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A1636AFCD7F34445A53E917E493DD1AF
2016-05-17 21:29:33,863 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F217EAD43FDD440299B9869285565B9A, ordinal: 0
2016-05-17 21:29:33,863 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F217EAD43FDD440299B9869285565B9A, mem on start (mb), free: 280, total: 356, max: 672
2016-05-17 21:29:33,864 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:33,864 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:33,864 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:33,864 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:33,864 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:33,864 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:33,864 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:33,864 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:33,865 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:33,865 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:33,865 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F6A557A45A4241DF9ACBB497FC9DFF35, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:33,865 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A1636AFCD7F34445A53E917E493DD1AF, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:33,865 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:33,865 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F6A557A45A4241DF9ACBB497FC9DFF35 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:33,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:33,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:33,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F6A557A45A4241DF9ACBB497FC9DFF35
2016-05-17 21:29:33,866 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:33,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F6A557A45A4241DF9ACBB497FC9DFF35
2016-05-17 21:29:33,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:33,868 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520573646_0001_1_00_000000_0
2016-05-17 21:29:33,873 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:33,873 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:33,873 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F6A557A45A4241DF9ACBB497FC9DFF35
2016-05-17 21:29:33,873 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: B6809A77B14240C88EF5D9A48EF87788
2016-05-17 21:29:33,874 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] B6809A77B14240C88EF5D9A48EF87788
2016-05-17 21:29:33,874 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3a6f80de
2016-05-17 21:29:33,875 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:33,876 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3a6f80de
2016-05-17 21:29:33,876 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:33,877 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:33,879 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:33,879 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@6e758cca; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@676e5a6d; comparator=cascading.tuple.hadoop.util.TupleComparator@7df2df27; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:33,879 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:33,880 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:33,880 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:33,893 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:33,893 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:33,893 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:33,894 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:33,894 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:33,895 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:33,895 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:33,896 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F217EAD43FDD440299B9869285565B9A, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:33,899 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520573646_0001_1_00_000000_0
2016-05-17 21:29:33,899 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F217EAD43FDD440299B9869285565B9A, mem on close (mb), free: 178, total: 356, max: 672
2016-05-17 21:29:33,899 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:33,900 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:33,900 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:33,900 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:33,906 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:33 Completed running task attempt: attempt_1463520573646_0001_1_00_000000_0
2016-05-17 21:29:33,910 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520573646_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:33,911 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520573646_0001_1_00, eventCount=1
2016-05-17 21:29:33,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520573896, Process_Duration=3, Process_End_Time=1463520573899, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:33,911 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F217EAD43FDD440299B9869285565B9A, taskAttemptId=attempt_1463520573646_0001_1_00_000000_0, startTime=1463520573846, finishTime=1463520573911, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520573896, Process_Duration=3, Process_End_Time=1463520573899, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:33,911 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:33,912 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520573646_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:33,912 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:33,913 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:33,913 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:33,913 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520573646_0001_1_00_000000_0
2016-05-17 21:29:33,915 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:33,915 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:TASK_FINISHED]: vertexName=F217EAD43FDD440299B9869285565B9A, taskId=task_1463520573646_0001_1_00_000000, startTime=1463520573846, finishTime=1463520573914, timeTaken=68, status=SUCCEEDED, successfulAttemptID=attempt_1463520573646_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520573896, Process_Duration=3, Process_End_Time=1463520573899, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:33,915 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520573646_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:33,914 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520573646_0001_00_000001
2016-05-17 21:29:33,915 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A] : 1
2016-05-17 21:29:33,916 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A]
2016-05-17 21:29:33,917 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:33,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:VERTEX_FINISHED]: vertexName=F217EAD43FDD440299B9869285565B9A, vertexId=vertex_1463520573646_0001_1_00, initRequestedTime=1463520573810, initedTime=1463520573834, startRequestedTime=1463520573835, startedTime=1463520573835, finishTime=1463520573916, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520573896, Process_Duration=3, Process_End_Time=1463520573899, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520573846, firstTasksToStart=[ task_1463520573646_0001_1_00_000000 ], lastTaskFinishTime=1463520573911, lastTasksToFinish=[ task_1463520573646_0001_1_00_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520573646_0001_1_00_000000 ], longestDurationTasks=[ task_1463520573646_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:33,917 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:33,917 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520573646_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:33,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520573646_0001_00_000001, stoppedTime=1463520573917, exitStatus=0
2016-05-17 21:29:33,917 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:33,918 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520573646_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:33,918 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF] attempt: attempt_1463520573646_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:33,918 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A1636AFCD7F34445A53E917E493DD1AF
2016-05-17 21:29:33,918 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF]
2016-05-17 21:29:33,919 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:33,919 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520573646_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:33,919 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:TASK_STARTED]: vertexName=A1636AFCD7F34445A53E917E493DD1AF, taskId=task_1463520573646_0001_1_01_000000, scheduledTime=1463520573919, launchTime=1463520573919
2016-05-17 21:29:33,919 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:33,920 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520573646_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:33,920 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF]
2016-05-17 21:29:33,920 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520573646_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:33,921 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [79D0A1DE833A4971A84754D3F43F776E/F29B55279ED849A9A4515F5DA6660A49] every/(1/1) ...yplatform/-1_ARGS_RESULTS, VertexName: A1636AFCD7F34445A53E917E493DD1AF, VertexParallelism: 1, TaskAttemptID:attempt_1463520573646_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F217EAD43FDD440299B9869285565B9A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=42C3DD000FBA4CA2B9755EA0FAE999F4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:33,921 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520573646_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:33,921 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:33,922 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520573646_0001_00_000002
2016-05-17 21:29:33,922 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520573646_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:33,923 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520573646_0001_00_000002
2016-05-17 21:29:33,923 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:33,924 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520573646_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:33,924 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:33,924 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520573646_0001_00_000002, launchTime=1463520573924
2016-05-17 21:29:33,925 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520573646_0001_00_000002
2016-05-17 21:29:33,926 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520573646_0001_1_01_000000_0] to container: [container_1463520573646_0001_00_000002]
2016-05-17 21:29:33,926 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520573646_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:33,926 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520573646_0001_00_000002 given task: attempt_1463520573646_0001_1_01_000000_0
2016-05-17 21:29:33,926 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520573646_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520573646_0001_1_01_000000_0
2016-05-17 21:29:33,926 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520573646_0001_1_01_000000_0] started. Is using containerId: [container_1463520573646_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:33,926 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:33 Starting to run new task attempt: attempt_1463520573646_0001_1_01_000000_0
2016-05-17 21:29:33,927 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520573646_0001_1_01_000000_0
2016-05-17 21:29:33,927 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:33,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A1636AFCD7F34445A53E917E493DD1AF, taskAttemptId=attempt_1463520573646_0001_1_01_000000_0, startTime=1463520573926, containerId=container_1463520573646_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520573646_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:33,928 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520573646_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:33,928 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520573646_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:33,928 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520573646_0001_00_000001, Exiting
2016-05-17 21:29:33,928 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520573646_0001_00_000001
2016-05-17 21:29:33,928 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520573646_0001_00_000001
2016-05-17 21:29:33,928 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:33,928 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520573646_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:33,929 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520573646_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:33,930 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:33,930 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:33,930 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:33,931 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:33,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [79D0A1DE833A4971A84754D3F43F776E/F29B55279ED849A9A4515F5DA6660A49] every/(1/1) ...yplatform/-1_ARGS_RESULTS, VertexName: A1636AFCD7F34445A53E917E493DD1AF, VertexParallelism: 1, TaskAttemptID:attempt_1463520573646_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F217EAD43FDD440299B9869285565B9A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=42C3DD000FBA4CA2B9755EA0FAE999F4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:33,930 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520573646_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:33,931 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520573646_0001_00_000001 completed successfully
2016-05-17 21:29:33,931 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:33,931 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520573646_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:33,932 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520573646_0001_1_01_000000_0
2016-05-17 21:29:33,932 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:33,932 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:33,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:33,933 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520573646_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:33,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=42C3DD000FBA4CA2B9755EA0FAE999F4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:33,934 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:33,934 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 42C3DD000FBA4CA2B9755EA0FAE999F4
2016-05-17 21:29:33,934 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:33,935 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 42C3DD000FBA4CA2B9755EA0FAE999F4
2016-05-17 21:29:33,934 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F217EAD43FDD440299B9869285565B9A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:33,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:33,936 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:33,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F217EAD43FDD440299B9869285565B9A
2016-05-17 21:29:33,938 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:33,938 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:33,938 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F217EAD43FDD440299B9869285565B9A
2016-05-17 21:29:33,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F217EAD43FDD440299B9869285565B9A
2016-05-17 21:29:33,944 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A1636AFCD7F34445A53E917E493DD1AF, ordinal: 1
2016-05-17 21:29:33,944 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1636AFCD7F34445A53E917E493DD1AF, mem on start (mb), free: 170, total: 356, max: 672
2016-05-17 21:29:33,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:33,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:33,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:33,945 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:33,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 42C3DD000FBA4CA2B9755EA0FAE999F4
2016-05-17 21:29:33,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:33,945 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:33,946 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:33,946 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:33,946 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:33,946 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:33,946 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:33,946 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 42C3DD000FBA4CA2B9755EA0FAE999F4, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:33,946 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F217EAD43FDD440299B9869285565B9A, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:33,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:33,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F217EAD43FDD440299B9869285565B9A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:33,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:33,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F217EAD43FDD440299B9869285565B9A
2016-05-17 21:29:33,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:33,948 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:33,948 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:33,948 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F217EAD43FDD440299B9869285565B9A with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:33,949 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:33,949 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:33,949 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:33,949 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F217EAD43FDD440299B9869285565B9A
2016-05-17 21:29:33,950 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:33,950 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520573646_0001_1_01_000000_0
2016-05-17 21:29:33,950 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520573646_0001_1_00_000000_0_10099, runDuration: 53129, ]
2016-05-17 21:29:33,956 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:33,956 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: B6809A77B14240C88EF5D9A48EF87788
2016-05-17 21:29:33,956 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_RESULTS"], id: 42C3DD000FBA4CA2B9755EA0FAE999F4
2016-05-17 21:29:33,957 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_RESULTS"]
2016-05-17 21:29:33,957 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] B6809A77B14240C88EF5D9A48EF87788, for 1 inputs
2016-05-17 21:29:33,958 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520573646_0001_1_00_000000_0_10099] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520573958, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:33,958 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F217EAD43FDD440299B9869285565B9A
2016-05-17 21:29:33,959 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:33,959 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [F217EAD43FDD440299B9869285565B9A] #17 in 1ms
2016-05-17 21:29:33,959 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F217EAD43FDD440299B9869285565B9A] #17, status:false, isInterrupted:false
2016-05-17 21:29:33,960 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:33,961 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:33,961 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:33,961 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:33,961 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:33,962 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:33,962 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F217EAD43FDD440299B9869285565B9A
2016-05-17 21:29:33,962 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:33,962 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A1636AFCD7F34445A53E917E493DD1AF, all 1 inputs ready in: 00:00:00.005
2016-05-17 21:29:33,964 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:33,966 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@7f1755c6; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@55007b59
2016-05-17 21:29:33,971 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:33,971 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:33,972 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520573646_0001_1_01_000000_0
2016-05-17 21:29:33,972 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520573646_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:33,973 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520573646_0001_1_01_000000_0
2016-05-17 21:29:33,973 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A1636AFCD7F34445A53E917E493DD1AF, mem on close (mb), free: 167, total: 356, max: 672
2016-05-17 21:29:33,973 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F217EAD43FDD440299B9869285565B9A
2016-05-17 21:29:33,973 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:33,975 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:33,975 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:33,975 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:33,975 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:33,976 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:33 Completed running task attempt: attempt_1463520573646_0001_1_01_000000_0
2016-05-17 21:29:33,977 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520573646_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:33,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520573963, Process_Duration=10, Process_End_Time=1463520573973, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:33,978 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:33,978 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A1636AFCD7F34445A53E917E493DD1AF, taskAttemptId=attempt_1463520573646_0001_1_01_000000_0, startTime=1463520573926, finishTime=1463520573978, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520573963, Process_Duration=10, Process_End_Time=1463520573973, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:33,979 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:33,979 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:33,979 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520573646_0001_00_000002
2016-05-17 21:29:33,979 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520573646_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:33,980 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:33,980 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520573646_0001_1_01_000000_0
2016-05-17 21:29:33,981 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:TASK_FINISHED]: vertexName=A1636AFCD7F34445A53E917E493DD1AF, taskId=task_1463520573646_0001_1_01_000000, startTime=1463520573926, finishTime=1463520573980, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1463520573646_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520573963, Process_Duration=10, Process_End_Time=1463520573973, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:33,981 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520573646_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:33,981 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF] : 1
2016-05-17 21:29:33,981 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF]
2016-05-17 21:29:33,982 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:VERTEX_FINISHED]: vertexName=A1636AFCD7F34445A53E917E493DD1AF, vertexId=vertex_1463520573646_0001_1_01, initRequestedTime=1463520573813, initedTime=1463520573830, startRequestedTime=1463520573837, startedTime=1463520573837, finishTime=1463520573981, timeTaken=144, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=373817344, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520573963, Process_Duration=10, Process_End_Time=1463520573973, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520573926, firstTasksToStart=[ task_1463520573646_0001_1_01_000000 ], lastTaskFinishTime=1463520573978, lastTasksToFinish=[ task_1463520573646_0001_1_01_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520573646_0001_1_01_000000 ], longestDurationTasks=[ task_1463520573646_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:33,982 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:33,982 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520573646_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:33,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520573646_0001_00_000002, stoppedTime=1463520573983, exitStatus=0
2016-05-17 21:29:33,983 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520573646_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:33,983 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:33,983 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520573646_0001_1
2016-05-17 21:29:33,983 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:33,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:33,985 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:33,986 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520573646_0001_1
2016-05-17 21:29:33,986 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520573646_0001_1_00 [F217EAD43FDD440299B9869285565B9A]
2016-05-17 21:29:33,986 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 42C3DD000FBA4CA2B9755EA0FAE999F4 for vertex: vertex_1463520573646_0001_1_01 [A1636AFCD7F34445A53E917E493DD1AF]
2016-05-17 21:29:33,990 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520573646_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:33,990 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520573646_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:33,991 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520573646_0001_00_000002, Exiting
2016-05-17 21:29:33,991 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520573646_0001_00_000002
2016-05-17 21:29:33,991 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520573646_0001_00_000002
2016-05-17 21:29:33,991 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520573646_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:33,991 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520573646_0001_00_000002 completed successfully
2016-05-17 21:29:33,998 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520573646_0001_1, queueSize=0
2016-05-17 21:29:33,999 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:34,000 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520573646_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520573646_0001_1, startTime=1463520573810, finishTime=1463520573996, timeTaken=186, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=345, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=747634688, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041147859, Process_Duration=13, Process_End_Time=2927041147872, Read_Duration=5, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:34,000 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520573646_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:34,000 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520573646_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:34,001 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520573646_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:34,001 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520573646_0001_00_000002
2016-05-17 21:29:34 Completed Dag: dag_1463520573646_0001_1
2016-05-17 21:29:34,002 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520573646_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:34,002 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520573646_0001_1_post
2016-05-17 21:29:34,002 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:34,002 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:34,008 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_ARGS_RESULTS, applicationId=application_1463520573646_0001
2016-05-17 21:29:34,010 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:34,010 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:34,010 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:34,010 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:34,010 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:34,011 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:34,011 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:34,011 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:34,011 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:34,012 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:34,012 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:34,012 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:34,013 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:34,019 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/79D0A1DE833A4971A84754D3F43F776E/F29B55279ED849A9A4515F5DA6660A49/.tez/application_1463520573646_0001
2016-05-17 21:29:34,019 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_RESULTS/_temporary
2016-05-17 21:29:34,020 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.393
2016-05-17 21:29:34,083 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_RESULTS
2016-05-17 21:29:34,102 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:34,103 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:34,127 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 21:29:34,130 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-05-17 21:29:34,131 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:34,132 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:34,132 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_ALL
2016-05-17 21:29:34,152 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:34,152 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:34,199 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 21:29:34,213 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-05-17 21:29:34,214 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:34,214 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:34,214 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:34,258 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:34,266 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_REPLACE
2016-05-17 21:29:34,325 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:34,325 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:34,345 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:34,345 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:34,346 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_REPLACE"]
2016-05-17 21:29:34,346 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:34,346 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:34,346 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:34,347 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...yplatform/-1_ARGS_REPLACE
2016-05-17 21:29:34,347 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:34,359 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:34,365 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:34,365 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:34,365 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:34,375 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1C5195222E96407BB1B354FA69B11DA2/DDEC64EFA44B4512B22A240760B041B5/.tez/application_1463520574365_0001 doesn't exist and is created
2016-05-17 21:29:34,392 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:34,392 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:34,393 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1C5195222E96407BB1B354FA69B11DA2/DDEC64EFA44B4512B22A240760B041B5/.tez/application_1463520574365_0001
2016-05-17 21:29:34,396 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520574365_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:34,405 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:34,406 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:34,408 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:34,408 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:34,408 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1C5195222E96407BB1B354FA69B11DA2/DDEC64EFA44B4512B22A240760B041B5/.tez/application_1463520574365_0001/localmode-log-dir
2016-05-17 21:29:34,408 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1C5195222E96407BB1B354FA69B11DA2/DDEC64EFA44B4512B22A240760B041B5/.tez/application_1463520574365_0001/localmode-log-dir/history.txt.appattempt_1463520574365_0001_000000, maxErrors=10
2016-05-17 21:29:34,408 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:34,408 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520574365_0001_000000, appSubmitTime=1463520574395, launchTime=1463520574395
2016-05-17 21:29:34,412 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:34,414 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:34,415 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:34,415 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:34,416 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:59711
2016-05-17 21:29:34,418 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:34,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520574365_0001_000000, startTime=1463520574421
2016-05-17 21:29:34,421 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:34,421 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:34,493 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:34,493 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:34,493 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_REPLACE, applicationId=application_1463520574365_0001, dagName=[1C5195222E96407BB1B354FA69B11DA2/DDEC64EFA44B4512B22A240760B041B5] every/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-05-17 21:29:34,509 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1C5195222E96407BB1B354FA69B11DA2/DDEC64EFA44B4512B22A240760B041B5] every/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-05-17 21:29:34,511 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520574365_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1C5195222E96407BB1B354FA69B11DA2/DDEC64EFA44B4512B22A240760B041B5/.tez/application_1463520574365_0001/localmode-log-dir/dag_1463520574365_0001_1.dot
2016-05-17 21:29:34,511 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1C5195222E96407BB1B354FA69B11DA2/DDEC64EFA44B4512B22A240760B041B5/.tez/application_1463520574365_0001/localmode-log-dir/dag_1463520574365_0001_1-tez-dag.pb.txt
2016-05-17 21:29:34,520 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520574365_0001_1
2016-05-17 21:29:34,521 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:34,521 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1C5195222E96407BB1B354FA69B11DA2/DDEC64EFA44B4512B22A240760B041B5] every/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-05-17 21:29:34 Running Dag: dag_1463520574365_0001_1
2016-05-17 21:29:34,536 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:34,537 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520574365_0001_1, submitTime=1463520574509
2016-05-17 21:29:34,538 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EEDB80CBB8EC4EBAA67D3073BACCFB54
2016-05-17 21:29:34,538 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:34,540 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:34,540 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520574365_0001_1, initTime=1463520574537
2016-05-17 21:29:34,540 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520574365_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:34,541 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:DAG_STARTED]: dagID=dag_1463520574365_0001_1, startTime=1463520574541
2016-05-17 21:29:34,541 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:34,541 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:34,541 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520574365_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:34,541 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B1EA33AE87C94C8F8E992732091AA618 : {7894B514B70D40D3928ADD5277745954={InputName=7894B514B70D40D3928ADD5277745954}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:34,541 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 7894B514B70D40D3928ADD5277745954, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:34,542 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618]
2016-05-17 21:29:34,542 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618]
2016-05-17 21:29:34,542 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618]
2016-05-17 21:29:34,542 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618]. Starting root input initializers: 1
2016-05-17 21:29:34,543 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:34,543 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54]
2016-05-17 21:29:34,546 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:34,548 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_REPLACE, applicationId=application_1463520574365_0001, dagName=[1C5195222E96407BB1B354FA69B11DA2/DDEC64EFA44B4512B22A240760B041B5] every/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-05-17 21:29:34,548 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54]
2016-05-17 21:29:34,549 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54]
2016-05-17 21:29:34,549 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54]
2016-05-17 21:29:34,550 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=E079397B2D1944049E96AC28B571AAB2, vertexId=vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:34,550 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=E079397B2D1944049E96AC28B571AAB2, vertexId=vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54]
2016-05-17 21:29:34,552 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=E079397B2D1944049E96AC28B571AAB2, vertexName=EEDB80CBB8EC4EBAA67D3073BACCFB54, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:34,553 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=E079397B2D1944049E96AC28B571AAB2, vertexId=vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54]
2016-05-17 21:29:34,554 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:34,555 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EEDB80CBB8EC4EBAA67D3073BACCFB54, vertexId=vertex_1463520574365_0001_1_01, initRequestedTime=1463520574543, initedTime=1463520574554, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:34,555 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:34,555 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 7894B514B70D40D3928ADD5277745954 on vertex vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618]
2016-05-17 21:29:34,557 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:34,558 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 7894B514B70D40D3928ADD5277745954 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:34,558 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:34,559 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:34,560 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:34,561 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 7894B514B70D40D3928ADD5277745954 on vertex vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618]
2016-05-17 21:29:34,561 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:34,562 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {7894B514B70D40D3928ADD5277745954=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:34,562 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618] parallelism set to 1
2016-05-17 21:29:34,563 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:34,563 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520574365_0001_1_00, eventCount=1
2016-05-17 21:29:34,564 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:34,564 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B1EA33AE87C94C8F8E992732091AA618, vertexId=vertex_1463520574365_0001_1_00, initRequestedTime=1463520574541, initedTime=1463520574564, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:34,564 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618] with distanceFromRoot: 0
2016-05-17 21:29:34,565 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:34,565 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B1EA33AE87C94C8F8E992732091AA618
2016-05-17 21:29:34,566 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618]
2016-05-17 21:29:34,566 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:34,567 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520574365_0001_1_00, startRequestedTime=1463520574565, startedTime=1463520574565
2016-05-17 21:29:34,567 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:34,568 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:34,568 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:TASK_STARTED]: vertexName=B1EA33AE87C94C8F8E992732091AA618, taskId=task_1463520574365_0001_1_00_000000, scheduledTime=1463520574567, launchTime=1463520574567
2016-05-17 21:29:34,568 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520574365_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:34,569 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520574365_0001_1_00 for vertex: vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54] numStartedSources: 1 numSources: 1
2016-05-17 21:29:34,569 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54] with distanceFromRoot: 1
2016-05-17 21:29:34,570 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618]
2016-05-17 21:29:34,570 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B1EA33AE87C94C8F8E992732091AA618 in vertex: EEDB80CBB8EC4EBAA67D3073BACCFB54
2016-05-17 21:29:34,571 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: EEDB80CBB8EC4EBAA67D3073BACCFB54 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:34,571 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:34,571 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520574365_0001_1_01, startRequestedTime=1463520574570, startedTime=1463520574570
2016-05-17 21:29:34,572 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:34,572 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1C5195222E96407BB1B354FA69B11DA2/DDEC64EFA44B4512B22A240760B041B5] every/(1/1) ...yplatform/-1_ARGS_REPLACE, VertexName: B1EA33AE87C94C8F8E992732091AA618, VertexParallelism: 1, TaskAttemptID:attempt_1463520574365_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7894B514B70D40D3928ADD5277745954, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EEDB80CBB8EC4EBAA67D3073BACCFB54, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:34,573 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520574365_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:34,573 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:34,582 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:34,583 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:34,583 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:34,584 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520574365_0001_00_000001
2016-05-17 21:29:34,584 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520574365_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:34,583 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520574365_0001, with dag id: dag_1463520574365_0001_1
2016-05-17 21:29:34,588 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520574365_0001_00_000001
2016-05-17 21:29:34,588 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:34,590 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520574365_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:34,592 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:34,594 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520574365_0001_00_000001, launchTime=1463520574590
2016-05-17 21:29:34,596 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520574365_0001_00_000001
2016-05-17 21:29:34,597 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520574365_0001_1_00_000000_0] to container: [container_1463520574365_0001_00_000001]
2016-05-17 21:29:34,597 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520574365_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:34,599 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520574365_0001_1_00_000000_0] started. Is using containerId: [container_1463520574365_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:34,599 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B1EA33AE87C94C8F8E992732091AA618, taskAttemptId=attempt_1463520574365_0001_1_00_000000_0, startTime=1463520574598, containerId=container_1463520574365_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520574365_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:34,599 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520574365_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:34,599 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520574365_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:34,600 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:34,601 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520574365_0001_00_000001 given task: attempt_1463520574365_0001_1_00_000000_0
2016-05-17 21:29:34,601 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520574365_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520574365_0001_1_00_000000_0
2016-05-17 21:29:34 Starting to run new task attempt: attempt_1463520574365_0001_1_00_000000_0
2016-05-17 21:29:34,601 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520574365_0001_1_00_000000_0
2016-05-17 21:29:34,602 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:34,602 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:34,602 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:34,602 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:34,602 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:34,602 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:34,602 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1C5195222E96407BB1B354FA69B11DA2/DDEC64EFA44B4512B22A240760B041B5] every/(1/1) ...yplatform/-1_ARGS_REPLACE, VertexName: B1EA33AE87C94C8F8E992732091AA618, VertexParallelism: 1, TaskAttemptID:attempt_1463520574365_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7894B514B70D40D3928ADD5277745954, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EEDB80CBB8EC4EBAA67D3073BACCFB54, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:34,602 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:34,603 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520574365_0001_1_00_000000_0
2016-05-17 21:29:34,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:34,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:34,603 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520574365_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:34,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:34,606 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:34,609 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EEDB80CBB8EC4EBAA67D3073BACCFB54, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:34,610 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:34,610 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EEDB80CBB8EC4EBAA67D3073BACCFB54
2016-05-17 21:29:34,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=7894B514B70D40D3928ADD5277745954, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:34,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:34,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 7894B514B70D40D3928ADD5277745954
2016-05-17 21:29:34,611 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 7894B514B70D40D3928ADD5277745954
2016-05-17 21:29:34,613 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:34,613 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EEDB80CBB8EC4EBAA67D3073BACCFB54
2016-05-17 21:29:34,614 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EEDB80CBB8EC4EBAA67D3073BACCFB54
2016-05-17 21:29:34,616 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:34,616 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 7894B514B70D40D3928ADD5277745954
2016-05-17 21:29:34,616 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 7894B514B70D40D3928ADD5277745954
2016-05-17 21:29:34,619 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B1EA33AE87C94C8F8E992732091AA618, ordinal: 0
2016-05-17 21:29:34,619 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1EA33AE87C94C8F8E992732091AA618, mem on start (mb), free: 260, total: 358, max: 672
2016-05-17 21:29:34,619 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:34,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:34,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:34,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:34,621 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:34,622 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:34,622 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:34,624 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:34,624 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:34,624 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:34,625 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 7894B514B70D40D3928ADD5277745954, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:34,625 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EEDB80CBB8EC4EBAA67D3073BACCFB54, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:34,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:34,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 7894B514B70D40D3928ADD5277745954
2016-05-17 21:29:34,626 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:34,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 7894B514B70D40D3928ADD5277745954
2016-05-17 21:29:34,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 7894B514B70D40D3928ADD5277745954 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:34,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:34,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:34,627 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:34,627 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520574365_0001_1_00_000000_0
2016-05-17 21:29:34,627 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:34,628 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@51e09b53
2016-05-17 21:29:34,629 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@51e09b53
2016-05-17 21:29:34,629 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:34,629 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:34,630 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:34,631 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 7894B514B70D40D3928ADD5277745954
2016-05-17 21:29:34,631 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 65E48E0B64FA4314ABCC2831C45151C5
2016-05-17 21:29:34,631 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 65E48E0B64FA4314ABCC2831C45151C5
2016-05-17 21:29:34,632 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:34,636 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:34,637 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@5afbee6; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@551df0f4; comparator=cascading.tuple.hadoop.util.TupleComparator@4301c74a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:34,637 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:34,637 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:34,638 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:34,651 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:34,652 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:34,652 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:34,652 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:34,653 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:34,653 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:34,654 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:34,654 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B1EA33AE87C94C8F8E992732091AA618, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:34,657 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520574365_0001_1_00_000000_0
2016-05-17 21:29:34,657 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1EA33AE87C94C8F8E992732091AA618, mem on close (mb), free: 158, total: 358, max: 672
2016-05-17 21:29:34,658 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:34,658 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:34,659 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:34,659 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:34,665 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:34 Completed running task attempt: attempt_1463520574365_0001_1_00_000000_0
2016-05-17 21:29:34,669 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520574365_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:34,670 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520574365_0001_1_00, eventCount=1
2016-05-17 21:29:34,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520574654, Process_Duration=3, Process_End_Time=1463520574657, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:34,671 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:34,670 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B1EA33AE87C94C8F8E992732091AA618, taskAttemptId=attempt_1463520574365_0001_1_00_000000_0, startTime=1463520574598, finishTime=1463520574670, timeTaken=72, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520574654, Process_Duration=3, Process_End_Time=1463520574657, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:34,673 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520574365_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:34,673 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:34,673 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520574365_0001_1_00_000000_0
2016-05-17 21:29:34,674 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:TASK_FINISHED]: vertexName=B1EA33AE87C94C8F8E992732091AA618, taskId=task_1463520574365_0001_1_00_000000, startTime=1463520574598, finishTime=1463520574673, timeTaken=75, status=SUCCEEDED, successfulAttemptID=attempt_1463520574365_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520574654, Process_Duration=3, Process_End_Time=1463520574657, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:34,674 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520574365_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:34,674 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618] : 1
2016-05-17 21:29:34,674 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618]
2016-05-17 21:29:34,675 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:34,675 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:VERTEX_FINISHED]: vertexName=B1EA33AE87C94C8F8E992732091AA618, vertexId=vertex_1463520574365_0001_1_00, initRequestedTime=1463520574541, initedTime=1463520574564, startRequestedTime=1463520574565, startedTime=1463520574565, finishTime=1463520574674, timeTaken=109, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520574654, Process_Duration=3, Process_End_Time=1463520574657, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520574598, firstTasksToStart=[ task_1463520574365_0001_1_00_000000 ], lastTaskFinishTime=1463520574670, lastTasksToFinish=[ task_1463520574365_0001_1_00_000000 ], minTaskDuration=72, maxTaskDuration=72, avgTaskDuration=72.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520574365_0001_1_00_000000 ], longestDurationTasks=[ task_1463520574365_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:34,676 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:34,676 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520574365_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:34,676 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54] attempt: attempt_1463520574365_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:34,676 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EEDB80CBB8EC4EBAA67D3073BACCFB54
2016-05-17 21:29:34,676 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54]
2016-05-17 21:29:34,677 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:34,676 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:34,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:TASK_STARTED]: vertexName=EEDB80CBB8EC4EBAA67D3073BACCFB54, taskId=task_1463520574365_0001_1_01_000000, scheduledTime=1463520574677, launchTime=1463520574677
2016-05-17 21:29:34,677 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520574365_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:34,677 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54]
2016-05-17 21:29:34,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520574365_0001_00_000001, stoppedTime=1463520574677, exitStatus=0
2016-05-17 21:29:34,678 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520574365_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:34,678 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1C5195222E96407BB1B354FA69B11DA2/DDEC64EFA44B4512B22A240760B041B5] every/(1/1) ...yplatform/-1_ARGS_REPLACE, VertexName: EEDB80CBB8EC4EBAA67D3073BACCFB54, VertexParallelism: 1, TaskAttemptID:attempt_1463520574365_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B1EA33AE87C94C8F8E992732091AA618, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E079397B2D1944049E96AC28B571AAB2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:34,678 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520574365_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:34,677 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:34,679 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:34,679 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:34,679 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520574365_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:34,679 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520574365_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:34,679 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520574365_0001_00_000002
2016-05-17 21:29:34,680 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520574365_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:34,681 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:34,681 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:34,681 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520574365_0001_00_000002
2016-05-17 21:29:34,681 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:34,682 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520574365_0001_00_000001
2016-05-17 21:29:34,682 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520574365_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:34,682 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520574365_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:34,683 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520574365_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:34,683 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520574365_0001_00_000001, Exiting
2016-05-17 21:29:34,683 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520574365_0001_00_000001
2016-05-17 21:29:34,683 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520574365_0001_00_000001
2016-05-17 21:29:34,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:34,684 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520574365_0001_00_000002, launchTime=1463520574683
2016-05-17 21:29:34,688 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520574365_0001_00_000002
2016-05-17 21:29:34,688 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520574365_0001_1_01_000000_0] to container: [container_1463520574365_0001_00_000002]
2016-05-17 21:29:34,688 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520574365_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:34,689 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520574365_0001_1_01_000000_0] started. Is using containerId: [container_1463520574365_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:34,689 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EEDB80CBB8EC4EBAA67D3073BACCFB54, taskAttemptId=attempt_1463520574365_0001_1_01_000000_0, startTime=1463520574689, containerId=container_1463520574365_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520574365_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:34,689 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520574365_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:34,690 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520574365_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:34,690 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:34,690 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520574365_0001_00_000002 given task: attempt_1463520574365_0001_1_01_000000_0
2016-05-17 21:29:34,691 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520574365_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520574365_0001_1_01_000000_0
2016-05-17 21:29:34 Starting to run new task attempt: attempt_1463520574365_0001_1_01_000000_0
2016-05-17 21:29:34,691 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520574365_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:34,691 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520574365_0001_00_000001 completed successfully
2016-05-17 21:29:34,691 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520574365_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:34,691 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520574365_0001_1_01_000000_0
2016-05-17 21:29:34,692 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:34,692 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:34,692 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:34,692 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:34,692 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:34,692 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:34,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1C5195222E96407BB1B354FA69B11DA2/DDEC64EFA44B4512B22A240760B041B5] every/(1/1) ...yplatform/-1_ARGS_REPLACE, VertexName: EEDB80CBB8EC4EBAA67D3073BACCFB54, VertexParallelism: 1, TaskAttemptID:attempt_1463520574365_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B1EA33AE87C94C8F8E992732091AA618, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E079397B2D1944049E96AC28B571AAB2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:34,693 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:34,693 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520574365_0001_1_01_000000_0
2016-05-17 21:29:34,693 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:34,696 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520574365_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:34,696 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:34,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:34,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B1EA33AE87C94C8F8E992732091AA618, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:34,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:34,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B1EA33AE87C94C8F8E992732091AA618
2016-05-17 21:29:34,699 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:34,699 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:34,699 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B1EA33AE87C94C8F8E992732091AA618
2016-05-17 21:29:34,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B1EA33AE87C94C8F8E992732091AA618
2016-05-17 21:29:34,700 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:34,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E079397B2D1944049E96AC28B571AAB2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:34,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:34,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E079397B2D1944049E96AC28B571AAB2
2016-05-17 21:29:34,705 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:34,705 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: E079397B2D1944049E96AC28B571AAB2
2016-05-17 21:29:34,707 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EEDB80CBB8EC4EBAA67D3073BACCFB54, ordinal: 1
2016-05-17 21:29:34,707 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EEDB80CBB8EC4EBAA67D3073BACCFB54, mem on start (mb), free: 151, total: 358, max: 672
2016-05-17 21:29:34,708 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:34,708 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:34,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:34,715 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:34,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E079397B2D1944049E96AC28B571AAB2
2016-05-17 21:29:34,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:34,716 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:34,716 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:34,716 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:34,716 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:34,716 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:34,717 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:34,717 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B1EA33AE87C94C8F8E992732091AA618, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:34,717 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E079397B2D1944049E96AC28B571AAB2, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:34,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:34,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B1EA33AE87C94C8F8E992732091AA618 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:34,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:34,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:34,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B1EA33AE87C94C8F8E992732091AA618
2016-05-17 21:29:34,718 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:34,718 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:34,719 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B1EA33AE87C94C8F8E992732091AA618 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:34,719 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:34,719 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:34,719 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:34,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B1EA33AE87C94C8F8E992732091AA618
2016-05-17 21:29:34,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:34,721 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520574365_0001_1_01_000000_0
2016-05-17 21:29:34,723 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520574365_0001_1_00_000000_0_10104, runDuration: 56447, ]
2016-05-17 21:29:34,724 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520574365_0001_1_00_000000_0_10104] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520574724, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:34,725 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B1EA33AE87C94C8F8E992732091AA618
2016-05-17 21:29:34,725 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:34,725 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:34,725 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 65E48E0B64FA4314ABCC2831C45151C5
2016-05-17 21:29:34,726 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_REPLACE"], id: E079397B2D1944049E96AC28B571AAB2
2016-05-17 21:29:34,727 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_REPLACE"]
2016-05-17 21:29:34,727 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 65E48E0B64FA4314ABCC2831C45151C5, for 1 inputs
2016-05-17 21:29:34,727 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B1EA33AE87C94C8F8E992732091AA618] #18 in 3ms
2016-05-17 21:29:34,728 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B1EA33AE87C94C8F8E992732091AA618] #18, status:false, isInterrupted:false
2016-05-17 21:29:34,728 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:34,729 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:34,729 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:34,729 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:34,729 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:34,730 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:34,730 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B1EA33AE87C94C8F8E992732091AA618
2016-05-17 21:29:34,730 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:34,730 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EEDB80CBB8EC4EBAA67D3073BACCFB54, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:34,731 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:34,733 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@3825350; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@612baafc
2016-05-17 21:29:34,736 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:34,736 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:34,736 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520574365_0001_1_01_000000_0
2016-05-17 21:29:34,737 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520574365_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:34,737 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520574365_0001_1_01_000000_0
2016-05-17 21:29:34,737 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EEDB80CBB8EC4EBAA67D3073BACCFB54, mem on close (mb), free: 147, total: 358, max: 672
2016-05-17 21:29:34,737 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B1EA33AE87C94C8F8E992732091AA618
2016-05-17 21:29:34,738 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:34,741 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:34,741 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:34,741 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:34,741 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:34,741 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:34 Completed running task attempt: attempt_1463520574365_0001_1_01_000000_0
2016-05-17 21:29:34,743 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520574365_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:34,744 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:34,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EEDB80CBB8EC4EBAA67D3073BACCFB54, taskAttemptId=attempt_1463520574365_0001_1_01_000000_0, startTime=1463520574689, finishTime=1463520574744, timeTaken=55, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520574730, Process_Duration=7, Process_End_Time=1463520574737, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:34,744 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520574365_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:34,745 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520574365_0001_1_01_000000_0
2016-05-17 21:29:34,745 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:34,745 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:34,745 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:TASK_FINISHED]: vertexName=EEDB80CBB8EC4EBAA67D3073BACCFB54, taskId=task_1463520574365_0001_1_01_000000, startTime=1463520574689, finishTime=1463520574745, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1463520574365_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520574730, Process_Duration=7, Process_End_Time=1463520574737, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:34,746 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520574365_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:34,746 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54] : 1
2016-05-17 21:29:34,746 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54]
2016-05-17 21:29:34,747 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:34,747 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:VERTEX_FINISHED]: vertexName=EEDB80CBB8EC4EBAA67D3073BACCFB54, vertexId=vertex_1463520574365_0001_1_01, initRequestedTime=1463520574543, initedTime=1463520574554, startRequestedTime=1463520574570, startedTime=1463520574570, finishTime=1463520574746, timeTaken=176, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520574730, Process_Duration=7, Process_End_Time=1463520574737, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520574689, firstTasksToStart=[ task_1463520574365_0001_1_01_000000 ], lastTaskFinishTime=1463520574744, lastTasksToFinish=[ task_1463520574365_0001_1_01_000000 ], minTaskDuration=55, maxTaskDuration=55, avgTaskDuration=55.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520574365_0001_1_01_000000 ], longestDurationTasks=[ task_1463520574365_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:34,747 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:34,748 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520574365_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:34,748 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520574365_0001_00_000002, stoppedTime=1463520574748, exitStatus=0
2016-05-17 21:29:34,748 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:34,748 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520574365_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:34,749 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:34,749 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520574365_0001_1
2016-05-17 21:29:34,749 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520574365_0001_1
2016-05-17 21:29:34,752 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520574365_0001_1_00 [B1EA33AE87C94C8F8E992732091AA618]
2016-05-17 21:29:34,752 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: E079397B2D1944049E96AC28B571AAB2 for vertex: vertex_1463520574365_0001_1_01 [EEDB80CBB8EC4EBAA67D3073BACCFB54]
2016-05-17 21:29:34,756 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520574730, Process_Duration=7, Process_End_Time=1463520574737, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:34,756 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:34,756 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:34,756 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520574365_0001_00_000002
2016-05-17 21:29:34,757 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520574365_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:34,757 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520574365_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:34,757 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520574365_0001_00_000002, Exiting
2016-05-17 21:29:34,757 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520574365_0001_00_000002
2016-05-17 21:29:34,757 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520574365_0001_00_000002
2016-05-17 21:29:34,758 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520574365_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:34,758 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520574365_0001_00_000002 completed successfully
2016-05-17 21:29:34,768 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520574365_0001_1, queueSize=0
2016-05-17 21:29:34,768 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:34,768 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520574365_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520574365_0001_1, startTime=1463520574541, finishTime=1463520574766, timeTaken=225, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=400, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=750780416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041149384, Process_Duration=10, Process_End_Time=2927041149394, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:34,769 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520574365_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:34,769 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520574365_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:34,777 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520574365_0001_00_000002
2016-05-17 21:29:34,777 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520574365_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:34 Completed Dag: dag_1463520574365_0001_1
2016-05-17 21:29:34,780 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520574365_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:34,780 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520574365_0001_1_post
2016-05-17 21:29:34,781 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:34,781 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:34,783 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_ARGS_REPLACE, applicationId=application_1463520574365_0001
2016-05-17 21:29:34,784 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:34,785 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:34,785 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:34,785 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:34,785 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:34,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:34,786 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:34,786 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:34,786 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:34,787 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:34,787 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:34,787 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:34,787 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:34,791 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1C5195222E96407BB1B354FA69B11DA2/DDEC64EFA44B4512B22A240760B041B5/.tez/application_1463520574365_0001
2016-05-17 21:29:34,792 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_REPLACE/_temporary
2016-05-17 21:29:34,792 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.447
2016-05-17 21:29:34,852 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_REPLACE
2016-05-17 21:29:34,872 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:34,873 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:34,922 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 21:29:34,929 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 21:29:34,929 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:34,930 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:34,930 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:34,930 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:34,930 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:34,943 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:34,949 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_SWAP
2016-05-17 21:29:34,990 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:34,991 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:34,991 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_SWAP"]
2016-05-17 21:29:34,991 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:34,991 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:34,991 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:34,992 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...mblyplatform/-1_ARGS_SWAP
2016-05-17 21:29:34,992 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:35,003 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:35,008 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:35,008 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:35,008 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:35,017 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/535E754C60AE4E6F95F32314DF4BFA31/DFCF047BBA104A32A85A26B421266D2A/.tez/application_1463520575008_0001 doesn't exist and is created
2016-05-17 21:29:35,031 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:35,032 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:35,032 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/535E754C60AE4E6F95F32314DF4BFA31/DFCF047BBA104A32A85A26B421266D2A/.tez/application_1463520575008_0001
2016-05-17 21:29:35,035 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520575008_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:35,044 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:35,044 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:35,046 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:35,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:35,046 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/535E754C60AE4E6F95F32314DF4BFA31/DFCF047BBA104A32A85A26B421266D2A/.tez/application_1463520575008_0001/localmode-log-dir
2016-05-17 21:29:35,046 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/535E754C60AE4E6F95F32314DF4BFA31/DFCF047BBA104A32A85A26B421266D2A/.tez/application_1463520575008_0001/localmode-log-dir/history.txt.appattempt_1463520575008_0001_000000, maxErrors=10
2016-05-17 21:29:35,047 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:35,047 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520575008_0001_000000, appSubmitTime=1463520575034, launchTime=1463520575034
2016-05-17 21:29:35,049 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:35,053 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:35,055 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:46829
2016-05-17 21:29:35,056 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:35,058 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:35,060 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:35,062 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520575008_0001_000000, startTime=1463520575062
2016-05-17 21:29:35,062 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:35,063 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:35,132 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:35,132 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:35,132 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_ARGS_SWAP, applicationId=application_1463520575008_0001, dagName=[535E754C60AE4E6F95F32314DF4BFA31/DFCF047BBA104A32A85A26B421266D2A] every/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-05-17 21:29:35,148 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [535E754C60AE4E6F95F32314DF4BFA31/DFCF047BBA104A32A85A26B421266D2A] every/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-05-17 21:29:35,149 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520575008_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/535E754C60AE4E6F95F32314DF4BFA31/DFCF047BBA104A32A85A26B421266D2A/.tez/application_1463520575008_0001/localmode-log-dir/dag_1463520575008_0001_1.dot
2016-05-17 21:29:35,150 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/535E754C60AE4E6F95F32314DF4BFA31/DFCF047BBA104A32A85A26B421266D2A/.tez/application_1463520575008_0001/localmode-log-dir/dag_1463520575008_0001_1-tez-dag.pb.txt
2016-05-17 21:29:35,157 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520575008_0001_1
2016-05-17 21:29:35,157 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:35,157 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [535E754C60AE4E6F95F32314DF4BFA31/DFCF047BBA104A32A85A26B421266D2A] every/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-05-17 21:29:35 Running Dag: dag_1463520575008_0001_1
2016-05-17 21:29:35,169 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520575008_0001_1, submitTime=1463520575148
2016-05-17 21:29:35,169 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:35,171 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F495B9510B1544E4B29FAFA4CEE93CCC
2016-05-17 21:29:35,171 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:35,171 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520575008_0001_1, initTime=1463520575170
2016-05-17 21:29:35,171 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:35,172 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520575008_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:35,173 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:DAG_STARTED]: dagID=dag_1463520575008_0001_1, startTime=1463520575173
2016-05-17 21:29:35,174 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:35,174 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520575008_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:35,174 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D33157CA23DC4C80B43D57D998B0EA15 : {0200911E34684CF585360EF71A8B3146={InputName=0200911E34684CF585360EF71A8B3146}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:35,174 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0200911E34684CF585360EF71A8B3146, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:35,174 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15]
2016-05-17 21:29:35,174 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15]
2016-05-17 21:29:35,174 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15]
2016-05-17 21:29:35,174 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15]. Starting root input initializers: 1
2016-05-17 21:29:35,174 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:35,175 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:35,175 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC]
2016-05-17 21:29:35,176 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_ARGS_SWAP, applicationId=application_1463520575008_0001, dagName=[535E754C60AE4E6F95F32314DF4BFA31/DFCF047BBA104A32A85A26B421266D2A] every/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-05-17 21:29:35,177 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:35,177 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC]
2016-05-17 21:29:35,177 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC]
2016-05-17 21:29:35,177 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC]
2016-05-17 21:29:35,177 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=82B3BF15BAE8414E9B0098689744DFBA, vertexId=vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:35,177 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=82B3BF15BAE8414E9B0098689744DFBA, vertexId=vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC]
2016-05-17 21:29:35,179 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=82B3BF15BAE8414E9B0098689744DFBA, vertexName=F495B9510B1544E4B29FAFA4CEE93CCC, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:35,179 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=82B3BF15BAE8414E9B0098689744DFBA, vertexId=vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC]
2016-05-17 21:29:35,179 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F495B9510B1544E4B29FAFA4CEE93CCC, vertexId=vertex_1463520575008_0001_1_01, initRequestedTime=1463520575175, initedTime=1463520575179, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:35,180 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:35,179 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:35,180 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0200911E34684CF585360EF71A8B3146 on vertex vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15]
2016-05-17 21:29:35,181 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:35,181 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0200911E34684CF585360EF71A8B3146 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:35,182 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:35,182 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:35,183 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:35,183 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0200911E34684CF585360EF71A8B3146 on vertex vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15]
2016-05-17 21:29:35,183 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:35,183 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0200911E34684CF585360EF71A8B3146=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:35,183 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15] parallelism set to 1
2016-05-17 21:29:35,184 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:35,184 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520575008_0001_1_00, eventCount=1
2016-05-17 21:29:35,184 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D33157CA23DC4C80B43D57D998B0EA15, vertexId=vertex_1463520575008_0001_1_00, initRequestedTime=1463520575174, initedTime=1463520575184, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:35,184 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15] with distanceFromRoot: 0
2016-05-17 21:29:35,184 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:35,184 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:35,185 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D33157CA23DC4C80B43D57D998B0EA15
2016-05-17 21:29:35,185 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15]
2016-05-17 21:29:35,185 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520575008_0001_1_00, startRequestedTime=1463520575184, startedTime=1463520575184
2016-05-17 21:29:35,185 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:35,185 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:35,186 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:35,186 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:TASK_STARTED]: vertexName=D33157CA23DC4C80B43D57D998B0EA15, taskId=task_1463520575008_0001_1_00_000000, scheduledTime=1463520575185, launchTime=1463520575185
2016-05-17 21:29:35,186 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520575008_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:35,186 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520575008_0001_1_00 for vertex: vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC] numStartedSources: 1 numSources: 1
2016-05-17 21:29:35,186 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC] with distanceFromRoot: 1
2016-05-17 21:29:35,186 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15]
2016-05-17 21:29:35,187 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D33157CA23DC4C80B43D57D998B0EA15 in vertex: F495B9510B1544E4B29FAFA4CEE93CCC
2016-05-17 21:29:35,187 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F495B9510B1544E4B29FAFA4CEE93CCC with 1 source tasks and 1 pending tasks
2016-05-17 21:29:35,187 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520575008_0001_1_01, startRequestedTime=1463520575187, startedTime=1463520575187
2016-05-17 21:29:35,187 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:35,187 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:35,188 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [535E754C60AE4E6F95F32314DF4BFA31/DFCF047BBA104A32A85A26B421266D2A] every/(1/1) ...mblyplatform/-1_ARGS_SWAP, VertexName: D33157CA23DC4C80B43D57D998B0EA15, VertexParallelism: 1, TaskAttemptID:attempt_1463520575008_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0200911E34684CF585360EF71A8B3146, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F495B9510B1544E4B29FAFA4CEE93CCC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:35,188 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520575008_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:35,188 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:35,189 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:35,189 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:35,190 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:35,190 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520575008_0001_00_000001
2016-05-17 21:29:35,190 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575008_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:35,191 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520575008_0001_00_000001
2016-05-17 21:29:35,191 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:35,192 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575008_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:35,192 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:35,192 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520575008_0001_00_000001, launchTime=1463520575192
2016-05-17 21:29:35,193 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520575008_0001_00_000001
2016-05-17 21:29:35,193 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520575008_0001, with dag id: dag_1463520575008_0001_1
2016-05-17 21:29:35,194 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520575008_0001_1_00_000000_0] to container: [container_1463520575008_0001_00_000001]
2016-05-17 21:29:35,194 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575008_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:35,195 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520575008_0001_00_000001 given task: attempt_1463520575008_0001_1_00_000000_0
2016-05-17 21:29:35,195 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520575008_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520575008_0001_1_00_000000_0
2016-05-17 21:29:35,195 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520575008_0001_1_00_000000_0] started. Is using containerId: [container_1463520575008_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:35,196 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:35 Starting to run new task attempt: attempt_1463520575008_0001_1_00_000000_0
2016-05-17 21:29:35,196 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520575008_0001_1_00_000000_0
2016-05-17 21:29:35,196 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D33157CA23DC4C80B43D57D998B0EA15, taskAttemptId=attempt_1463520575008_0001_1_00_000000_0, startTime=1463520575195, containerId=container_1463520575008_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520575008_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:35,196 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520575008_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:35,197 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520575008_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:35,197 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:35,197 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:35,197 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:35,197 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:35,198 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:35,198 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:35,198 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [535E754C60AE4E6F95F32314DF4BFA31/DFCF047BBA104A32A85A26B421266D2A] every/(1/1) ...mblyplatform/-1_ARGS_SWAP, VertexName: D33157CA23DC4C80B43D57D998B0EA15, VertexParallelism: 1, TaskAttemptID:attempt_1463520575008_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0200911E34684CF585360EF71A8B3146, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F495B9510B1544E4B29FAFA4CEE93CCC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:35,198 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:35,199 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520575008_0001_1_00_000000_0
2016-05-17 21:29:35,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:35,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:35,200 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0200911E34684CF585360EF71A8B3146, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:35,203 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520575008_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:35,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:35,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F495B9510B1544E4B29FAFA4CEE93CCC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:35,200 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:35,205 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0200911E34684CF585360EF71A8B3146
2016-05-17 21:29:35,206 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0200911E34684CF585360EF71A8B3146
2016-05-17 21:29:35,206 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:35,206 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F495B9510B1544E4B29FAFA4CEE93CCC
2016-05-17 21:29:35,208 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:35,208 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0200911E34684CF585360EF71A8B3146
2016-05-17 21:29:35,208 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:35,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0200911E34684CF585360EF71A8B3146
2016-05-17 21:29:35,209 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:35,209 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F495B9510B1544E4B29FAFA4CEE93CCC
2016-05-17 21:29:35,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F495B9510B1544E4B29FAFA4CEE93CCC
2016-05-17 21:29:35,213 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D33157CA23DC4C80B43D57D998B0EA15, ordinal: 0
2016-05-17 21:29:35,213 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D33157CA23DC4C80B43D57D998B0EA15, mem on start (mb), free: 259, total: 358, max: 672
2016-05-17 21:29:35,213 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:35,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:35,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:35,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:35,214 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:35,214 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:35,214 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:35,214 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:35,215 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:35,215 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:35,215 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0200911E34684CF585360EF71A8B3146, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:35,215 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F495B9510B1544E4B29FAFA4CEE93CCC, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:35,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:35,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0200911E34684CF585360EF71A8B3146 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:35,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:35,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:35,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0200911E34684CF585360EF71A8B3146
2016-05-17 21:29:35,216 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:35,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0200911E34684CF585360EF71A8B3146
2016-05-17 21:29:35,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:35,216 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520575008_0001_1_00_000000_0
2016-05-17 21:29:35,217 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:35,217 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@74b40f23
2016-05-17 21:29:35,218 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@74b40f23
2016-05-17 21:29:35,218 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:35,219 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:35,219 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:35,220 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0200911E34684CF585360EF71A8B3146
2016-05-17 21:29:35,220 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: F6C0B4696836463CAC0EE45BE389C049
2016-05-17 21:29:35,220 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] F6C0B4696836463CAC0EE45BE389C049
2016-05-17 21:29:35,221 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:35,223 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:35,223 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3ab80d25; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@9671641; comparator=cascading.tuple.hadoop.util.TupleComparator@60a94ca6; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:35,224 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:35,224 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:35,225 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:35,237 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:35,238 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:35,238 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:35,238 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:35,238 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:35,239 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:35,239 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:35,240 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D33157CA23DC4C80B43D57D998B0EA15, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:35,242 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520575008_0001_1_00_000000_0
2016-05-17 21:29:35,242 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D33157CA23DC4C80B43D57D998B0EA15, mem on close (mb), free: 156, total: 358, max: 672
2016-05-17 21:29:35,242 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:35,242 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:35,243 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:35,243 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:35,246 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:35 Completed running task attempt: attempt_1463520575008_0001_1_00_000000_0
2016-05-17 21:29:35,249 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520575008_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:35,250 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520575008_0001_1_00, eventCount=1
2016-05-17 21:29:35,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520575240, Process_Duration=2, Process_End_Time=1463520575242, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:35,251 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:35,251 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:35,251 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:35,251 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D33157CA23DC4C80B43D57D998B0EA15, taskAttemptId=attempt_1463520575008_0001_1_00_000000_0, startTime=1463520575195, finishTime=1463520575250, timeTaken=55, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520575240, Process_Duration=2, Process_End_Time=1463520575242, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:35,252 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520575008_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:35,252 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:35,252 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520575008_0001_1_00_000000_0
2016-05-17 21:29:35,252 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520575008_0001_00_000001
2016-05-17 21:29:35,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:35,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:TASK_FINISHED]: vertexName=D33157CA23DC4C80B43D57D998B0EA15, taskId=task_1463520575008_0001_1_00_000000, startTime=1463520575195, finishTime=1463520575252, timeTaken=57, status=SUCCEEDED, successfulAttemptID=attempt_1463520575008_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520575240, Process_Duration=2, Process_End_Time=1463520575242, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:35,253 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520575008_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:35,253 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15] : 1
2016-05-17 21:29:35,253 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15]
2016-05-17 21:29:35,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:35,254 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:VERTEX_FINISHED]: vertexName=D33157CA23DC4C80B43D57D998B0EA15, vertexId=vertex_1463520575008_0001_1_00, initRequestedTime=1463520575174, initedTime=1463520575184, startRequestedTime=1463520575184, startedTime=1463520575184, finishTime=1463520575253, timeTaken=69, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520575240, Process_Duration=2, Process_End_Time=1463520575242, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520575195, firstTasksToStart=[ task_1463520575008_0001_1_00_000000 ], lastTaskFinishTime=1463520575250, lastTasksToFinish=[ task_1463520575008_0001_1_00_000000 ], minTaskDuration=55, maxTaskDuration=55, avgTaskDuration=55.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520575008_0001_1_00_000000 ], longestDurationTasks=[ task_1463520575008_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:35,255 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:35,255 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575008_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:35,255 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520575008_0001_00_000001, stoppedTime=1463520575255, exitStatus=0
2016-05-17 21:29:35,255 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:35,255 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575008_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:35,256 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC] attempt: attempt_1463520575008_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:35,256 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F495B9510B1544E4B29FAFA4CEE93CCC
2016-05-17 21:29:35,256 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC]
2016-05-17 21:29:35,257 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:35,257 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520575008_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:35,257 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:TASK_STARTED]: vertexName=F495B9510B1544E4B29FAFA4CEE93CCC, taskId=task_1463520575008_0001_1_01_000000, scheduledTime=1463520575257, launchTime=1463520575257
2016-05-17 21:29:35,257 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:35,257 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520575008_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:35,258 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC]
2016-05-17 21:29:35,258 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575008_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:35,258 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [535E754C60AE4E6F95F32314DF4BFA31/DFCF047BBA104A32A85A26B421266D2A] every/(1/1) ...mblyplatform/-1_ARGS_SWAP, VertexName: F495B9510B1544E4B29FAFA4CEE93CCC, VertexParallelism: 1, TaskAttemptID:attempt_1463520575008_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D33157CA23DC4C80B43D57D998B0EA15, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=82B3BF15BAE8414E9B0098689744DFBA, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:35,259 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520575008_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:35,259 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:35,259 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520575008_0001_00_000002
2016-05-17 21:29:35,259 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575008_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:35,261 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520575008_0001_00_000002
2016-05-17 21:29:35,261 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:35,265 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520575008_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:35,265 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520575008_0001_00_000001: 13 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:35,266 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520575008_0001_00_000001, Exiting
2016-05-17 21:29:35,266 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520575008_0001_00_000001
2016-05-17 21:29:35,266 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520575008_0001_00_000001
2016-05-17 21:29:35,267 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575008_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:35,268 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:35,268 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520575008_0001_00_000002, launchTime=1463520575267
2016-05-17 21:29:35,269 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520575008_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:35,269 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520575008_0001_00_000001 completed successfully
2016-05-17 21:29:35,270 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575008_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:35,270 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520575008_0001_00_000002
2016-05-17 21:29:35,270 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520575008_0001_1_01_000000_0] to container: [container_1463520575008_0001_00_000002]
2016-05-17 21:29:35,271 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575008_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:35,271 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520575008_0001_1_01_000000_0] started. Is using containerId: [container_1463520575008_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:35,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F495B9510B1544E4B29FAFA4CEE93CCC, taskAttemptId=attempt_1463520575008_0001_1_01_000000_0, startTime=1463520575271, containerId=container_1463520575008_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520575008_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:35,271 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520575008_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:35,272 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520575008_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:35,272 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:35,272 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520575008_0001_00_000002 given task: attempt_1463520575008_0001_1_01_000000_0
2016-05-17 21:29:35,272 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520575008_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520575008_0001_1_01_000000_0
2016-05-17 21:29:35 Starting to run new task attempt: attempt_1463520575008_0001_1_01_000000_0
2016-05-17 21:29:35,273 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520575008_0001_1_01_000000_0
2016-05-17 21:29:35,273 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:35,273 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:35,273 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:35,273 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:35,273 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:35,273 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:35,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [535E754C60AE4E6F95F32314DF4BFA31/DFCF047BBA104A32A85A26B421266D2A] every/(1/1) ...mblyplatform/-1_ARGS_SWAP, VertexName: F495B9510B1544E4B29FAFA4CEE93CCC, VertexParallelism: 1, TaskAttemptID:attempt_1463520575008_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D33157CA23DC4C80B43D57D998B0EA15, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=82B3BF15BAE8414E9B0098689744DFBA, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:35,274 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:35,274 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520575008_0001_1_01_000000_0
2016-05-17 21:29:35,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:35,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:35,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D33157CA23DC4C80B43D57D998B0EA15, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:35,276 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520575008_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:35,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=82B3BF15BAE8414E9B0098689744DFBA, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:35,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:35,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:35,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 82B3BF15BAE8414E9B0098689744DFBA
2016-05-17 21:29:35,278 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:35,278 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 82B3BF15BAE8414E9B0098689744DFBA
2016-05-17 21:29:35,280 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:35,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:35,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D33157CA23DC4C80B43D57D998B0EA15
2016-05-17 21:29:35,282 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:35,284 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:35,284 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D33157CA23DC4C80B43D57D998B0EA15
2016-05-17 21:29:35,285 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D33157CA23DC4C80B43D57D998B0EA15
2016-05-17 21:29:35,290 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F495B9510B1544E4B29FAFA4CEE93CCC, ordinal: 1
2016-05-17 21:29:35,290 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F495B9510B1544E4B29FAFA4CEE93CCC, mem on start (mb), free: 149, total: 358, max: 672
2016-05-17 21:29:35,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:35,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:35,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:35,292 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:35,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 82B3BF15BAE8414E9B0098689744DFBA
2016-05-17 21:29:35,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:35,292 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:35,293 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:35,293 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:35,293 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:35,293 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:35,294 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:35,294 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 82B3BF15BAE8414E9B0098689744DFBA, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:35,294 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D33157CA23DC4C80B43D57D998B0EA15, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:35,294 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:35,294 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D33157CA23DC4C80B43D57D998B0EA15
2016-05-17 21:29:35,295 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:35,295 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:35,296 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D33157CA23DC4C80B43D57D998B0EA15 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:35,296 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:35,296 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:35,296 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:35,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D33157CA23DC4C80B43D57D998B0EA15
2016-05-17 21:29:35,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D33157CA23DC4C80B43D57D998B0EA15 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:35,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:35,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:35,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:35,299 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520575008_0001_1_01_000000_0
2016-05-17 21:29:35,302 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520575008_0001_1_00_000000_0_10111, runDuration: 41386, ]
2016-05-17 21:29:35,302 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:35,303 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: F6C0B4696836463CAC0EE45BE389C049
2016-05-17 21:29:35,303 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_SWAP"], id: 82B3BF15BAE8414E9B0098689744DFBA
2016-05-17 21:29:35,304 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_SWAP"]
2016-05-17 21:29:35,304 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] F6C0B4696836463CAC0EE45BE389C049, for 1 inputs
2016-05-17 21:29:35,305 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520575008_0001_1_00_000000_0_10111] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520575305, TimeTaken=2, Rate=0.09 MB/s
2016-05-17 21:29:35,305 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D33157CA23DC4C80B43D57D998B0EA15
2016-05-17 21:29:35,305 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:35,306 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D33157CA23DC4C80B43D57D998B0EA15] #19 in 3ms
2016-05-17 21:29:35,306 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D33157CA23DC4C80B43D57D998B0EA15] #19, status:false, isInterrupted:false
2016-05-17 21:29:35,307 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:35,308 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:35,308 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:35,309 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:35,309 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:35,309 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:35,309 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D33157CA23DC4C80B43D57D998B0EA15
2016-05-17 21:29:35,309 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:35,310 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F495B9510B1544E4B29FAFA4CEE93CCC, all 1 inputs ready in: 00:00:00.006
2016-05-17 21:29:35,314 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:35,316 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@37cfd06c; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@2c7c7f14
2016-05-17 21:29:35,321 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:35,321 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:35,322 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520575008_0001_1_01_000000_0
2016-05-17 21:29:35,322 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520575008_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:35,323 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520575008_0001_1_01_000000_0
2016-05-17 21:29:35,323 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F495B9510B1544E4B29FAFA4CEE93CCC, mem on close (mb), free: 146, total: 358, max: 672
2016-05-17 21:29:35,324 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D33157CA23DC4C80B43D57D998B0EA15
2016-05-17 21:29:35,324 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:35,326 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:35,326 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:35,326 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:35,326 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:35,326 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:35 Completed running task attempt: attempt_1463520575008_0001_1_01_000000_0
2016-05-17 21:29:35,327 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520575008_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:35,328 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520575310, Process_Duration=13, Process_End_Time=1463520575323, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:35,329 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F495B9510B1544E4B29FAFA4CEE93CCC, taskAttemptId=attempt_1463520575008_0001_1_01_000000_0, startTime=1463520575271, finishTime=1463520575328, timeTaken=57, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520575310, Process_Duration=13, Process_End_Time=1463520575323, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:35,329 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:35,329 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:35,329 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:35,329 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520575008_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:35,329 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520575008_0001_1_01_000000_0
2016-05-17 21:29:35,330 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:35,330 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:TASK_FINISHED]: vertexName=F495B9510B1544E4B29FAFA4CEE93CCC, taskId=task_1463520575008_0001_1_01_000000, startTime=1463520575271, finishTime=1463520575330, timeTaken=59, status=SUCCEEDED, successfulAttemptID=attempt_1463520575008_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520575310, Process_Duration=13, Process_End_Time=1463520575323, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:35,330 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520575008_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:35,330 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC] : 1
2016-05-17 21:29:35,331 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC]
2016-05-17 21:29:35,331 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:35,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:VERTEX_FINISHED]: vertexName=F495B9510B1544E4B29FAFA4CEE93CCC, vertexId=vertex_1463520575008_0001_1_01, initRequestedTime=1463520575175, initedTime=1463520575179, startRequestedTime=1463520575187, startedTime=1463520575187, finishTime=1463520575331, timeTaken=144, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520575310, Process_Duration=13, Process_End_Time=1463520575323, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520575271, firstTasksToStart=[ task_1463520575008_0001_1_01_000000 ], lastTaskFinishTime=1463520575328, lastTasksToFinish=[ task_1463520575008_0001_1_01_000000 ], minTaskDuration=57, maxTaskDuration=57, avgTaskDuration=57.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520575008_0001_1_01_000000 ], longestDurationTasks=[ task_1463520575008_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:35,332 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:35,332 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:35,332 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:35,333 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520575008_0001_1
2016-05-17 21:29:35,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520575008_0001_1
2016-05-17 21:29:35,334 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520575008_0001_1_00 [D33157CA23DC4C80B43D57D998B0EA15]
2016-05-17 21:29:35,334 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 82B3BF15BAE8414E9B0098689744DFBA for vertex: vertex_1463520575008_0001_1_01 [F495B9510B1544E4B29FAFA4CEE93CCC]
2016-05-17 21:29:35,339 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520575008_0001_00_000002
2016-05-17 21:29:35,345 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520575008_0001_1, queueSize=0
2016-05-17 21:29:35,345 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:35,345 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520575008_0001_1, startTime=1463520575173, finishTime=1463520575343, timeTaken=170, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=400, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041150550, Process_Duration=15, Process_End_Time=2927041150565, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:35,345 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520575008_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:35,345 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520575008_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:35,346 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575008_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:35,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575008_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520575008_0001_00_000002, stoppedTime=1463520575346, exitStatus=0
2016-05-17 21:29:35,346 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575008_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:35,347 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:35 Completed Dag: dag_1463520575008_0001_1
2016-05-17 21:29:35,348 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520575008_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:35,348 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520575008_0001_1_post
2016-05-17 21:29:35,348 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:35,349 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:35,349 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520575008_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:35,350 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575008_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:35,350 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520575008_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:35,350 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520575008_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:35,350 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520575008_0001_00_000002, Exiting
2016-05-17 21:29:35,350 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520575008_0001_00_000002
2016-05-17 21:29:35,351 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520575008_0001_00_000002
2016-05-17 21:29:35,351 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520575008_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:35,351 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520575008_0001_00_000002 completed successfully
2016-05-17 21:29:35,351 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575008_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:35,353 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/-1_ARGS_SWAP, applicationId=application_1463520575008_0001
2016-05-17 21:29:35,355 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:35,355 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:35,355 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:35,355 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:35,355 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:35,355 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:35,356 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:35,356 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:35,356 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:35,356 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:35,356 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:35,357 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:35,357 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:35,360 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/535E754C60AE4E6F95F32314DF4BFA31/DFCF047BBA104A32A85A26B421266D2A/.tez/application_1463520575008_0001
2016-05-17 21:29:35,361 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_SWAP/_temporary
2016-05-17 21:29:35,361 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.371
2016-05-17 21:29:35,424 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_SWAP
2016-05-17 21:29:35,453 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:35,453 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:35,500 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:29:35,514 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-05-17 21:29:35,514 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:35,515 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:35,515 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:35,530 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:35,536 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_-1
2016-05-17 21:29:35,579 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:35,579 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:35,579 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_-1"]
2016-05-17 21:29:35,579 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:35,579 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:35,580 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:35,580 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...mblyplatform/0_'field'_-1
2016-05-17 21:29:35,580 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:35,591 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:35,595 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:35,596 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:35,596 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:35,605 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3E06AB9D92AA4BB78D8D510EEB0EB103/C6D7E5D2E930443AA0215465C8CD6824/.tez/application_1463520575596_0001 doesn't exist and is created
2016-05-17 21:29:35,609 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:35,609 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:35,621 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:35,621 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:35,622 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3E06AB9D92AA4BB78D8D510EEB0EB103/C6D7E5D2E930443AA0215465C8CD6824/.tez/application_1463520575596_0001
2016-05-17 21:29:35,625 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520575596_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:35,634 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:35,635 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:35,636 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:35,636 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:35,636 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3E06AB9D92AA4BB78D8D510EEB0EB103/C6D7E5D2E930443AA0215465C8CD6824/.tez/application_1463520575596_0001/localmode-log-dir
2016-05-17 21:29:35,637 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3E06AB9D92AA4BB78D8D510EEB0EB103/C6D7E5D2E930443AA0215465C8CD6824/.tez/application_1463520575596_0001/localmode-log-dir/history.txt.appattempt_1463520575596_0001_000000, maxErrors=10
2016-05-17 21:29:35,637 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:35,637 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520575596_0001_000000, appSubmitTime=1463520575625, launchTime=1463520575625
2016-05-17 21:29:35,640 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:59122
2016-05-17 21:29:35,642 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:35,643 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:35,644 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:35,645 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:35,646 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:35,648 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520575596_0001_000000, startTime=1463520575648
2016-05-17 21:29:35,648 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:35,648 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:35,722 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:35,722 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:35,722 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/0_'field'_-1, applicationId=application_1463520575596_0001, dagName=[3E06AB9D92AA4BB78D8D510EEB0EB103/C6D7E5D2E930443AA0215465C8CD6824] every/(1/1) ...mblyplatform/0_'field'_-1
2016-05-17 21:29:35,736 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3E06AB9D92AA4BB78D8D510EEB0EB103/C6D7E5D2E930443AA0215465C8CD6824] every/(1/1) ...mblyplatform/0_'field'_-1
2016-05-17 21:29:35,737 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520575596_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3E06AB9D92AA4BB78D8D510EEB0EB103/C6D7E5D2E930443AA0215465C8CD6824/.tez/application_1463520575596_0001/localmode-log-dir/dag_1463520575596_0001_1.dot
2016-05-17 21:29:35,738 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3E06AB9D92AA4BB78D8D510EEB0EB103/C6D7E5D2E930443AA0215465C8CD6824/.tez/application_1463520575596_0001/localmode-log-dir/dag_1463520575596_0001_1-tez-dag.pb.txt
2016-05-17 21:29:35,744 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520575596_0001_1
2016-05-17 21:29:35,744 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:35,744 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3E06AB9D92AA4BB78D8D510EEB0EB103/C6D7E5D2E930443AA0215465C8CD6824] every/(1/1) ...mblyplatform/0_'field'_-1
2016-05-17 21:29:35 Running Dag: dag_1463520575596_0001_1
2016-05-17 21:29:35,754 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520575596_0001_1, submitTime=1463520575736
2016-05-17 21:29:35,754 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:35,755 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E4488B04388348638CF81CB56B596F44
2016-05-17 21:29:35,755 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:35,756 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520575596_0001_1, initTime=1463520575754
2016-05-17 21:29:35,756 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520575596_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:35,756 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:35,757 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:DAG_STARTED]: dagID=dag_1463520575596_0001_1, startTime=1463520575756
2016-05-17 21:29:35,757 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:35,757 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:35,757 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520575596_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:35,757 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CAC00C47332D4707B2135471ACFF6978 : {7E6EDA517FDC42DD908CFC30C0FFA860={InputName=7E6EDA517FDC42DD908CFC30C0FFA860}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:35,757 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 7E6EDA517FDC42DD908CFC30C0FFA860, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:35,757 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978]
2016-05-17 21:29:35,757 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978]
2016-05-17 21:29:35,757 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978]
2016-05-17 21:29:35,758 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978]. Starting root input initializers: 1
2016-05-17 21:29:35,758 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:35,758 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44]
2016-05-17 21:29:35,758 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 7E6EDA517FDC42DD908CFC30C0FFA860 on vertex vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978]
2016-05-17 21:29:35,760 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:35,760 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 7E6EDA517FDC42DD908CFC30C0FFA860 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:35,760 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/0_'field'_-1, applicationId=application_1463520575596_0001, dagName=[3E06AB9D92AA4BB78D8D510EEB0EB103/C6D7E5D2E930443AA0215465C8CD6824] every/(1/1) ...mblyplatform/0_'field'_-1
2016-05-17 21:29:35,761 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:35,761 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:35,761 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:35,761 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44]
2016-05-17 21:29:35,761 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44]
2016-05-17 21:29:35,762 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44]
2016-05-17 21:29:35,762 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=CEF153F4CC8C4E2691958F9B3CBD1552, vertexId=vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:35,762 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=CEF153F4CC8C4E2691958F9B3CBD1552, vertexId=vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44]
2016-05-17 21:29:35,763 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:35,763 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 7E6EDA517FDC42DD908CFC30C0FFA860 on vertex vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978]
2016-05-17 21:29:35,764 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=CEF153F4CC8C4E2691958F9B3CBD1552, vertexName=E4488B04388348638CF81CB56B596F44, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:35,764 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=CEF153F4CC8C4E2691958F9B3CBD1552, vertexId=vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44]
2016-05-17 21:29:35,765 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:35,766 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E4488B04388348638CF81CB56B596F44, vertexId=vertex_1463520575596_0001_1_01, initRequestedTime=1463520575758, initedTime=1463520575765, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:35,766 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:35,766 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:35,766 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {7E6EDA517FDC42DD908CFC30C0FFA860=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:35,767 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978] parallelism set to 1
2016-05-17 21:29:35,767 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:35,767 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520575596_0001_1_00, eventCount=1
2016-05-17 21:29:35,768 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:35,768 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CAC00C47332D4707B2135471ACFF6978, vertexId=vertex_1463520575596_0001_1_00, initRequestedTime=1463520575757, initedTime=1463520575768, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:35,768 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978] with distanceFromRoot: 0
2016-05-17 21:29:35,768 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:35,769 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CAC00C47332D4707B2135471ACFF6978
2016-05-17 21:29:35,769 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978]
2016-05-17 21:29:35,769 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520575596_0001_1_00, startRequestedTime=1463520575769, startedTime=1463520575769
2016-05-17 21:29:35,769 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:35,770 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:35,770 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:35,770 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:TASK_STARTED]: vertexName=CAC00C47332D4707B2135471ACFF6978, taskId=task_1463520575596_0001_1_00_000000, scheduledTime=1463520575770, launchTime=1463520575770
2016-05-17 21:29:35,771 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520575596_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:35,771 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520575596_0001_1_00 for vertex: vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44] numStartedSources: 1 numSources: 1
2016-05-17 21:29:35,771 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44] with distanceFromRoot: 1
2016-05-17 21:29:35,771 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978]
2016-05-17 21:29:35,772 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CAC00C47332D4707B2135471ACFF6978 in vertex: E4488B04388348638CF81CB56B596F44
2016-05-17 21:29:35,772 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E4488B04388348638CF81CB56B596F44 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:35,773 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:35,773 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520575596_0001_1_01, startRequestedTime=1463520575772, startedTime=1463520575772
2016-05-17 21:29:35,773 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:35,773 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3E06AB9D92AA4BB78D8D510EEB0EB103/C6D7E5D2E930443AA0215465C8CD6824] every/(1/1) ...mblyplatform/0_'field'_-1, VertexName: CAC00C47332D4707B2135471ACFF6978, VertexParallelism: 1, TaskAttemptID:attempt_1463520575596_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7E6EDA517FDC42DD908CFC30C0FFA860, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E4488B04388348638CF81CB56B596F44, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:35,774 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520575596_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:35,774 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:35,774 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:35,775 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:35,775 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:35,776 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520575596_0001_00_000001
2016-05-17 21:29:35,776 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575596_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:35,777 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520575596_0001_00_000001
2016-05-17 21:29:35,777 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:35,778 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575596_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:35,778 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:35,778 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520575596_0001_00_000001, launchTime=1463520575778
2016-05-17 21:29:35,779 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520575596_0001_00_000001
2016-05-17 21:29:35,779 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520575596_0001_1_00_000000_0] to container: [container_1463520575596_0001_00_000001]
2016-05-17 21:29:35,780 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575596_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:35,780 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520575596_0001_1_00_000000_0] started. Is using containerId: [container_1463520575596_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:35,780 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520575596_0001, with dag id: dag_1463520575596_0001_1
2016-05-17 21:29:35,780 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CAC00C47332D4707B2135471ACFF6978, taskAttemptId=attempt_1463520575596_0001_1_00_000000_0, startTime=1463520575780, containerId=container_1463520575596_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520575596_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:35,782 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520575596_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:35,782 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520575596_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:35,782 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:35,783 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520575596_0001_00_000001 given task: attempt_1463520575596_0001_1_00_000000_0
2016-05-17 21:29:35,783 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520575596_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520575596_0001_1_00_000000_0
2016-05-17 21:29:35 Starting to run new task attempt: attempt_1463520575596_0001_1_00_000000_0
2016-05-17 21:29:35,783 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520575596_0001_1_00_000000_0
2016-05-17 21:29:35,783 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:35,784 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:35,784 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:35,784 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:35,784 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:35,784 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:35,784 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3E06AB9D92AA4BB78D8D510EEB0EB103/C6D7E5D2E930443AA0215465C8CD6824] every/(1/1) ...mblyplatform/0_'field'_-1, VertexName: CAC00C47332D4707B2135471ACFF6978, VertexParallelism: 1, TaskAttemptID:attempt_1463520575596_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7E6EDA517FDC42DD908CFC30C0FFA860, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E4488B04388348638CF81CB56B596F44, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:35,784 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:35,785 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520575596_0001_1_00_000000_0
2016-05-17 21:29:35,785 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:35,785 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:35,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=7E6EDA517FDC42DD908CFC30C0FFA860, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:35,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:35,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E4488B04388348638CF81CB56B596F44, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:35,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:35,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E4488B04388348638CF81CB56B596F44
2016-05-17 21:29:35,789 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520575596_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:35,790 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:35,790 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:35,791 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 7E6EDA517FDC42DD908CFC30C0FFA860
2016-05-17 21:29:35,791 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 7E6EDA517FDC42DD908CFC30C0FFA860
2016-05-17 21:29:35,790 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:35,791 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E4488B04388348638CF81CB56B596F44
2016-05-17 21:29:35,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E4488B04388348638CF81CB56B596F44
2016-05-17 21:29:35,794 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:35,795 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 7E6EDA517FDC42DD908CFC30C0FFA860
2016-05-17 21:29:35,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 7E6EDA517FDC42DD908CFC30C0FFA860
2016-05-17 21:29:35,797 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CAC00C47332D4707B2135471ACFF6978, ordinal: 0
2016-05-17 21:29:35,797 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAC00C47332D4707B2135471ACFF6978, mem on start (mb), free: 255, total: 357, max: 672
2016-05-17 21:29:35,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:35,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:35,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:35,798 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:35,798 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:35,798 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:35,798 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:35,798 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:35,798 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:35,798 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:35,799 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 7E6EDA517FDC42DD908CFC30C0FFA860, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:35,799 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E4488B04388348638CF81CB56B596F44, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:35,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:35,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 7E6EDA517FDC42DD908CFC30C0FFA860 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:35,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:35,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:35,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 7E6EDA517FDC42DD908CFC30C0FFA860
2016-05-17 21:29:35,799 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:35,800 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 7E6EDA517FDC42DD908CFC30C0FFA860
2016-05-17 21:29:35,800 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:35,800 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520575596_0001_1_00_000000_0
2016-05-17 21:29:35,800 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:35,801 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@76fa0c78
2016-05-17 21:29:35,804 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@76fa0c78
2016-05-17 21:29:35,805 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:35,805 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:35,805 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:35,805 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 7E6EDA517FDC42DD908CFC30C0FFA860
2016-05-17 21:29:35,806 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 2531FFB652264EB584D0783E439B9CED
2016-05-17 21:29:35,806 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 2531FFB652264EB584D0783E439B9CED
2016-05-17 21:29:35,807 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:35,809 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:35,809 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@53deac96; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@4014d6de; comparator=cascading.tuple.hadoop.util.TupleComparator@1826a9b0; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:35,810 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:35,810 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:35,811 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:35,825 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:35,825 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:35,825 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:35,825 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:35,825 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:35,828 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:35,829 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:35,829 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CAC00C47332D4707B2135471ACFF6978, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:35,832 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520575596_0001_1_00_000000_0
2016-05-17 21:29:35,832 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAC00C47332D4707B2135471ACFF6978, mem on close (mb), free: 152, total: 357, max: 672
2016-05-17 21:29:35,832 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:35,832 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:35,832 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:35,832 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:35,840 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:35 Completed running task attempt: attempt_1463520575596_0001_1_00_000000_0
2016-05-17 21:29:35,844 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520575596_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:35,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520575596_0001_1_00, eventCount=1
2016-05-17 21:29:35,846 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:35,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CAC00C47332D4707B2135471ACFF6978, taskAttemptId=attempt_1463520575596_0001_1_00_000000_0, startTime=1463520575780, finishTime=1463520575846, timeTaken=66, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520575829, Process_Duration=2, Process_End_Time=1463520575831, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:35,847 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520575596_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:35,847 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:35,847 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520575596_0001_1_00_000000_0
2016-05-17 21:29:35,849 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:TASK_FINISHED]: vertexName=CAC00C47332D4707B2135471ACFF6978, taskId=task_1463520575596_0001_1_00_000000, startTime=1463520575780, finishTime=1463520575847, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1463520575596_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520575829, Process_Duration=2, Process_End_Time=1463520575831, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:35,849 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520575596_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:35,850 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:35,850 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978] : 1
2016-05-17 21:29:35,850 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978]
2016-05-17 21:29:35,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:VERTEX_FINISHED]: vertexName=CAC00C47332D4707B2135471ACFF6978, vertexId=vertex_1463520575596_0001_1_00, initRequestedTime=1463520575757, initedTime=1463520575768, startRequestedTime=1463520575769, startedTime=1463520575769, finishTime=1463520575850, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520575829, Process_Duration=2, Process_End_Time=1463520575831, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520575780, firstTasksToStart=[ task_1463520575596_0001_1_00_000000 ], lastTaskFinishTime=1463520575846, lastTasksToFinish=[ task_1463520575596_0001_1_00_000000 ], minTaskDuration=66, maxTaskDuration=66, avgTaskDuration=66.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520575596_0001_1_00_000000 ], longestDurationTasks=[ task_1463520575596_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:35,852 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:35,852 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:35,853 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575596_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:35,853 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520575596_0001_00_000001, stoppedTime=1463520575853, exitStatus=0
2016-05-17 21:29:35,853 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575596_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:35,853 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44] attempt: attempt_1463520575596_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:35,853 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E4488B04388348638CF81CB56B596F44
2016-05-17 21:29:35,854 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44]
2016-05-17 21:29:35,854 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:35,854 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520575596_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:35,854 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:35,855 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:35,855 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:TASK_STARTED]: vertexName=E4488B04388348638CF81CB56B596F44, taskId=task_1463520575596_0001_1_01_000000, scheduledTime=1463520575854, launchTime=1463520575854
2016-05-17 21:29:35,855 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520575596_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:35,856 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575596_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:35,856 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44]
2016-05-17 21:29:35,856 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3E06AB9D92AA4BB78D8D510EEB0EB103/C6D7E5D2E930443AA0215465C8CD6824] every/(1/1) ...mblyplatform/0_'field'_-1, VertexName: E4488B04388348638CF81CB56B596F44, VertexParallelism: 1, TaskAttemptID:attempt_1463520575596_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CAC00C47332D4707B2135471ACFF6978, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CEF153F4CC8C4E2691958F9B3CBD1552, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:35,857 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520575596_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:35,857 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:35,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520575829, Process_Duration=2, Process_End_Time=1463520575831, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:35,858 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520575596_0001_00_000002
2016-05-17 21:29:35,858 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:35,859 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:35,859 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575596_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:35,859 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520575596_0001_00_000001
2016-05-17 21:29:35,860 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520575596_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:35,860 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520575596_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:35,860 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520575596_0001_00_000001, Exiting
2016-05-17 21:29:35,860 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520575596_0001_00_000001
2016-05-17 21:29:35,860 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520575596_0001_00_000001
2016-05-17 21:29:35,861 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520575596_0001_00_000002
2016-05-17 21:29:35,861 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520575596_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:35,861 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:35,862 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520575596_0001_00_000001 completed successfully
2016-05-17 21:29:35,862 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575596_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:35,864 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575596_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:35,864 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:35,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520575596_0001_00_000002, launchTime=1463520575864
2016-05-17 21:29:35,865 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520575596_0001_00_000002
2016-05-17 21:29:35,865 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520575596_0001_1_01_000000_0] to container: [container_1463520575596_0001_00_000002]
2016-05-17 21:29:35,865 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575596_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:35,866 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520575596_0001_1_01_000000_0] started. Is using containerId: [container_1463520575596_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:35,866 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E4488B04388348638CF81CB56B596F44, taskAttemptId=attempt_1463520575596_0001_1_01_000000_0, startTime=1463520575866, containerId=container_1463520575596_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520575596_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:35,866 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520575596_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:35,866 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520575596_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:35,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:35,867 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520575596_0001_00_000002 given task: attempt_1463520575596_0001_1_01_000000_0
2016-05-17 21:29:35,867 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520575596_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520575596_0001_1_01_000000_0
2016-05-17 21:29:35 Starting to run new task attempt: attempt_1463520575596_0001_1_01_000000_0
2016-05-17 21:29:35,868 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520575596_0001_1_01_000000_0
2016-05-17 21:29:35,868 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:35,868 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:35,868 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:35,868 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:35,869 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:35,869 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:35,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3E06AB9D92AA4BB78D8D510EEB0EB103/C6D7E5D2E930443AA0215465C8CD6824] every/(1/1) ...mblyplatform/0_'field'_-1, VertexName: E4488B04388348638CF81CB56B596F44, VertexParallelism: 1, TaskAttemptID:attempt_1463520575596_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CAC00C47332D4707B2135471ACFF6978, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CEF153F4CC8C4E2691958F9B3CBD1552, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:35,869 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:35,870 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520575596_0001_1_01_000000_0
2016-05-17 21:29:35,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:35,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:35,872 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520575596_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:35,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CAC00C47332D4707B2135471ACFF6978, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:35,873 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:35,873 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CAC00C47332D4707B2135471ACFF6978
2016-05-17 21:29:35,874 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:35,875 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:35,875 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CAC00C47332D4707B2135471ACFF6978
2016-05-17 21:29:35,875 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CAC00C47332D4707B2135471ACFF6978
2016-05-17 21:29:35,875 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:35,878 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:35,886 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E4488B04388348638CF81CB56B596F44, ordinal: 1
2016-05-17 21:29:35,887 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4488B04388348638CF81CB56B596F44, mem on start (mb), free: 146, total: 357, max: 672
2016-05-17 21:29:35,887 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:35,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:35,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:35,887 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CEF153F4CC8C4E2691958F9B3CBD1552, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:35,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:35,889 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CEF153F4CC8C4E2691958F9B3CBD1552
2016-05-17 21:29:35,889 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:35,889 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: CEF153F4CC8C4E2691958F9B3CBD1552
2016-05-17 21:29:35,899 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:35,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CEF153F4CC8C4E2691958F9B3CBD1552
2016-05-17 21:29:35,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:35,900 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:35,900 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:35,900 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:35,900 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:35,900 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:35,901 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:35,901 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CAC00C47332D4707B2135471ACFF6978, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:35,901 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CEF153F4CC8C4E2691958F9B3CBD1552, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:35,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:35,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CAC00C47332D4707B2135471ACFF6978 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:35,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:35,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CAC00C47332D4707B2135471ACFF6978
2016-05-17 21:29:35,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:35,902 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:35,903 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:35,904 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CAC00C47332D4707B2135471ACFF6978 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:35,904 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:35,904 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:35,904 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:35,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CAC00C47332D4707B2135471ACFF6978
2016-05-17 21:29:35,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:35,912 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520575596_0001_1_01_000000_0
2016-05-17 21:29:35,915 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520575596_0001_1_00_000000_0_10116, runDuration: 53974, ]
2016-05-17 21:29:35,915 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:35,916 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 2531FFB652264EB584D0783E439B9CED
2016-05-17 21:29:35,916 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_-1"], id: CEF153F4CC8C4E2691958F9B3CBD1552
2016-05-17 21:29:35,917 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_-1"]
2016-05-17 21:29:35,917 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 2531FFB652264EB584D0783E439B9CED, for 1 inputs
2016-05-17 21:29:35,921 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520575596_0001_1_00_000000_0_10116] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520575921, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:35,921 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CAC00C47332D4707B2135471ACFF6978
2016-05-17 21:29:35,922 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:35,922 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [CAC00C47332D4707B2135471ACFF6978] #20 in 2ms
2016-05-17 21:29:35,923 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CAC00C47332D4707B2135471ACFF6978] #20, status:false, isInterrupted:false
2016-05-17 21:29:35,924 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:35,925 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:35,925 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:35,925 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:35,925 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:35,926 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:35,926 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CAC00C47332D4707B2135471ACFF6978
2016-05-17 21:29:35,926 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:35,926 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E4488B04388348638CF81CB56B596F44, all 1 inputs ready in: 00:00:00.009
2016-05-17 21:29:35,928 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:35,930 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@4ee5cb1c; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@5079c3bb
2016-05-17 21:29:35,936 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:35,936 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:35,937 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520575596_0001_1_01_000000_0
2016-05-17 21:29:35,937 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520575596_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:35,937 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520575596_0001_1_01_000000_0
2016-05-17 21:29:35,938 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4488B04388348638CF81CB56B596F44, mem on close (mb), free: 139, total: 357, max: 672
2016-05-17 21:29:35,938 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CAC00C47332D4707B2135471ACFF6978
2016-05-17 21:29:35,939 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:35,940 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:35,940 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:35,940 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:35,940 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:35,940 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:35 Completed running task attempt: attempt_1463520575596_0001_1_01_000000_0
2016-05-17 21:29:35,941 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520575596_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:35,942 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E4488B04388348638CF81CB56B596F44, taskAttemptId=attempt_1463520575596_0001_1_01_000000_0, startTime=1463520575866, finishTime=1463520575942, timeTaken=76, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=19, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=16, LAST_EVENT_RECEIVED=16, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520575927, Process_Duration=10, Process_End_Time=1463520575937, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:35,943 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520575596_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:35,943 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520575596_0001_1_01_000000_0
2016-05-17 21:29:35,943 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:TASK_FINISHED]: vertexName=E4488B04388348638CF81CB56B596F44, taskId=task_1463520575596_0001_1_01_000000, startTime=1463520575866, finishTime=1463520575943, timeTaken=77, status=SUCCEEDED, successfulAttemptID=attempt_1463520575596_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=19, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=16, LAST_EVENT_RECEIVED=16, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520575927, Process_Duration=10, Process_End_Time=1463520575937, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:35,943 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520575596_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:35,944 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44] : 1
2016-05-17 21:29:35,944 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:35,944 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44]
2016-05-17 21:29:35,944 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:35,945 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:35,945 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:VERTEX_FINISHED]: vertexName=E4488B04388348638CF81CB56B596F44, vertexId=vertex_1463520575596_0001_1_01, initRequestedTime=1463520575758, initedTime=1463520575765, startRequestedTime=1463520575772, startedTime=1463520575772, finishTime=1463520575944, timeTaken=172, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=19, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=16, LAST_EVENT_RECEIVED=16, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520575927, Process_Duration=10, Process_End_Time=1463520575937, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520575866, firstTasksToStart=[ task_1463520575596_0001_1_01_000000 ], lastTaskFinishTime=1463520575942, lastTasksToFinish=[ task_1463520575596_0001_1_01_000000 ], minTaskDuration=76, maxTaskDuration=76, avgTaskDuration=76.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520575596_0001_1_01_000000 ], longestDurationTasks=[ task_1463520575596_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:35,945 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:35,945 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:35,945 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575596_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:35,945 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520575596_0001_00_000002, stoppedTime=1463520575945, exitStatus=0
2016-05-17 21:29:35,945 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575596_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:35,946 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:35,946 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520575596_0001_1
2016-05-17 21:29:35,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:35,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520575596_0001_1
2016-05-17 21:29:35,946 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: CEF153F4CC8C4E2691958F9B3CBD1552 for vertex: vertex_1463520575596_0001_1_01 [E4488B04388348638CF81CB56B596F44]
2016-05-17 21:29:35,949 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=374865920, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=19, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=16, LAST_EVENT_RECEIVED=16][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520575927, Process_Duration=10, Process_End_Time=1463520575937, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:35,950 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:35,950 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:35,950 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520575596_0001_00_000002
2016-05-17 21:29:35,950 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520575596_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:35,950 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520575596_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:35,950 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520575596_0001_00_000002, Exiting
2016-05-17 21:29:35,950 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520575596_0001_00_000002
2016-05-17 21:29:35,950 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520575596_0001_00_000002
2016-05-17 21:29:35,951 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520575596_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:35,951 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520575596_0001_00_000002 completed successfully
2016-05-17 21:29:35,953 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520575596_0001_1_00 [CAC00C47332D4707B2135471ACFF6978]
2016-05-17 21:29:35,956 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520575596_0001_1, queueSize=0
2016-05-17 21:29:35,956 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:35,957 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520575596_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520575596_0001_1, startTime=1463520575756, finishTime=1463520575954, timeTaken=198, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=345, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=749731840, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=19, MERGE_PHASE_TIME=23, FIRST_EVENT_RECEIVED=16, LAST_EVENT_RECEIVED=16, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041151756, Process_Duration=12, Process_End_Time=2927041151768, Read_Duration=6, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:35,957 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520575596_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:35,957 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520575596_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:35,957 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520575596_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:35,957 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520575596_0001_00_000002
2016-05-17 21:29:35 Completed Dag: dag_1463520575596_0001_1
2016-05-17 21:29:35,957 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520575596_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:35,957 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520575596_0001_1_post
2016-05-17 21:29:35,958 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:35,958 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:35,964 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/0_'field'_-1, applicationId=application_1463520575596_0001
2016-05-17 21:29:35,965 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:35,965 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:35,965 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:35,965 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:35,966 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:35,966 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:35,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:35,966 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:35,967 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:35,967 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:35,967 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:35,967 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:35,968 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:35,974 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/3E06AB9D92AA4BB78D8D510EEB0EB103/C6D7E5D2E930443AA0215465C8CD6824/.tez/application_1463520575596_0001
2016-05-17 21:29:35,975 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_-1/_temporary
2016-05-17 21:29:35,975 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.396
2016-05-17 21:29:36,038 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_-1
2016-05-17 21:29:36,078 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:36,078 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:36,113 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 21:29:36,117 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-17 21:29:36,118 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:36,118 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:36,119 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:36,132 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:36,138 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_'field'
2016-05-17 21:29:36,181 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:36,181 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:36,181 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_'field'"]
2016-05-17 21:29:36,181 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:36,181 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:36,181 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:36,182 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...latform/0_'field'_'field'
2016-05-17 21:29:36,183 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:36,195 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:36,198 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:36,198 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:36,199 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:36,200 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:36,200 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:36,210 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2DF36936CD2A4DEDA003185B6AEE8035/BE18C468009F46F1989AF30C8D911A5B/.tez/application_1463520576200_0001 doesn't exist and is created
2016-05-17 21:29:36,226 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:36,227 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:36,227 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2DF36936CD2A4DEDA003185B6AEE8035/BE18C468009F46F1989AF30C8D911A5B/.tez/application_1463520576200_0001
2016-05-17 21:29:36,230 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520576200_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:36,239 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:36,240 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:36,241 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:36,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:36,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2DF36936CD2A4DEDA003185B6AEE8035/BE18C468009F46F1989AF30C8D911A5B/.tez/application_1463520576200_0001/localmode-log-dir
2016-05-17 21:29:36,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2DF36936CD2A4DEDA003185B6AEE8035/BE18C468009F46F1989AF30C8D911A5B/.tez/application_1463520576200_0001/localmode-log-dir/history.txt.appattempt_1463520576200_0001_000000, maxErrors=10
2016-05-17 21:29:36,242 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:36,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520576200_0001_000000, appSubmitTime=1463520576229, launchTime=1463520576229
2016-05-17 21:29:36,245 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:36,246 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:36,250 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:36,250 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:33132
2016-05-17 21:29:36,250 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:36,252 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:36,254 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520576200_0001_000000, startTime=1463520576254
2016-05-17 21:29:36,254 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:36,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:36,327 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:36,327 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:36,327 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_'field'_'field', applicationId=application_1463520576200_0001, dagName=[2DF36936CD2A4DEDA003185B6AEE8035/BE18C468009F46F1989AF30C8D911A5B] every/(1/1) ...latform/0_'field'_'field'
2016-05-17 21:29:36,341 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2DF36936CD2A4DEDA003185B6AEE8035/BE18C468009F46F1989AF30C8D911A5B] every/(1/1) ...latform/0_'field'_'field'
2016-05-17 21:29:36,343 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520576200_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2DF36936CD2A4DEDA003185B6AEE8035/BE18C468009F46F1989AF30C8D911A5B/.tez/application_1463520576200_0001/localmode-log-dir/dag_1463520576200_0001_1.dot
2016-05-17 21:29:36,344 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2DF36936CD2A4DEDA003185B6AEE8035/BE18C468009F46F1989AF30C8D911A5B/.tez/application_1463520576200_0001/localmode-log-dir/dag_1463520576200_0001_1-tez-dag.pb.txt
2016-05-17 21:29:36,350 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520576200_0001_1
2016-05-17 21:29:36,351 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:36,351 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2DF36936CD2A4DEDA003185B6AEE8035/BE18C468009F46F1989AF30C8D911A5B] every/(1/1) ...latform/0_'field'_'field'
2016-05-17 21:29:36 Running Dag: dag_1463520576200_0001_1
2016-05-17 21:29:36,363 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:36,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520576200_0001_1, submitTime=1463520576342
2016-05-17 21:29:36,365 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A69B866D1BC247D5922FF938591F777C
2016-05-17 21:29:36,365 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:36,366 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:36,366 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520576200_0001_1, initTime=1463520576364
2016-05-17 21:29:36,367 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520576200_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:36,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:DAG_STARTED]: dagID=dag_1463520576200_0001_1, startTime=1463520576367
2016-05-17 21:29:36,367 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:36,367 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520576200_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:36,367 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B36A1AE2CD3E4E22B0605AA7275E1431 : {A074FC18D84B49569DB0E847D37BB0D8={InputName=A074FC18D84B49569DB0E847D37BB0D8}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:36,367 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A074FC18D84B49569DB0E847D37BB0D8, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:36,367 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431]
2016-05-17 21:29:36,368 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431]
2016-05-17 21:29:36,368 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431]
2016-05-17 21:29:36,367 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:36,368 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431]. Starting root input initializers: 1
2016-05-17 21:29:36,369 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_'field'_'field', applicationId=application_1463520576200_0001, dagName=[2DF36936CD2A4DEDA003185B6AEE8035/BE18C468009F46F1989AF30C8D911A5B] every/(1/1) ...latform/0_'field'_'field'
2016-05-17 21:29:36,370 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:36,370 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C]
2016-05-17 21:29:36,371 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A074FC18D84B49569DB0E847D37BB0D8 on vertex vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431]
2016-05-17 21:29:36,373 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:36,373 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A074FC18D84B49569DB0E847D37BB0D8 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:36,374 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:36,374 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:36,375 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:36,375 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C]
2016-05-17 21:29:36,376 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C]
2016-05-17 21:29:36,376 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C]
2016-05-17 21:29:36,376 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=05ACBD6B31BD4DEC9940682EA834E7B7, vertexId=vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:36,376 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=05ACBD6B31BD4DEC9940682EA834E7B7, vertexId=vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C]
2016-05-17 21:29:36,377 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:36,377 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=05ACBD6B31BD4DEC9940682EA834E7B7, vertexName=A69B866D1BC247D5922FF938591F777C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:36,378 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=05ACBD6B31BD4DEC9940682EA834E7B7, vertexId=vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C]
2016-05-17 21:29:36,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:36,377 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A074FC18D84B49569DB0E847D37BB0D8 on vertex vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431]
2016-05-17 21:29:36,379 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A69B866D1BC247D5922FF938591F777C, vertexId=vertex_1463520576200_0001_1_01, initRequestedTime=1463520576370, initedTime=1463520576378, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:36,379 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:36,379 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:36,379 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A074FC18D84B49569DB0E847D37BB0D8=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:36,379 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431] parallelism set to 1
2016-05-17 21:29:36,379 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:36,379 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520576200_0001_1_00, eventCount=1
2016-05-17 21:29:36,380 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:36,380 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B36A1AE2CD3E4E22B0605AA7275E1431, vertexId=vertex_1463520576200_0001_1_00, initRequestedTime=1463520576367, initedTime=1463520576380, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:36,380 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431] with distanceFromRoot: 0
2016-05-17 21:29:36,380 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:36,380 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B36A1AE2CD3E4E22B0605AA7275E1431
2016-05-17 21:29:36,381 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431]
2016-05-17 21:29:36,381 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:36,381 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520576200_0001_1_00, startRequestedTime=1463520576380, startedTime=1463520576380
2016-05-17 21:29:36,381 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:36,382 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:36,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:TASK_STARTED]: vertexName=B36A1AE2CD3E4E22B0605AA7275E1431, taskId=task_1463520576200_0001_1_00_000000, scheduledTime=1463520576381, launchTime=1463520576381
2016-05-17 21:29:36,382 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520576200_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:36,382 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520576200_0001_1_00 for vertex: vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C] numStartedSources: 1 numSources: 1
2016-05-17 21:29:36,382 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C] with distanceFromRoot: 1
2016-05-17 21:29:36,382 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431]
2016-05-17 21:29:36,383 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B36A1AE2CD3E4E22B0605AA7275E1431 in vertex: A69B866D1BC247D5922FF938591F777C
2016-05-17 21:29:36,383 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A69B866D1BC247D5922FF938591F777C with 1 source tasks and 1 pending tasks
2016-05-17 21:29:36,383 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:36,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520576200_0001_1_01, startRequestedTime=1463520576383, startedTime=1463520576383
2016-05-17 21:29:36,384 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:36,384 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2DF36936CD2A4DEDA003185B6AEE8035/BE18C468009F46F1989AF30C8D911A5B] every/(1/1) ...latform/0_'field'_'field', VertexName: B36A1AE2CD3E4E22B0605AA7275E1431, VertexParallelism: 1, TaskAttemptID:attempt_1463520576200_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A074FC18D84B49569DB0E847D37BB0D8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A69B866D1BC247D5922FF938591F777C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:36,384 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520576200_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:36,384 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:36,385 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:36,385 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:36,385 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:36,385 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520576200_0001_00_000001
2016-05-17 21:29:36,385 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576200_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:36,387 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520576200_0001_00_000001
2016-05-17 21:29:36,387 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:36,388 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576200_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:36,388 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:36,388 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520576200_0001_00_000001, launchTime=1463520576388
2016-05-17 21:29:36,388 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520576200_0001, with dag id: dag_1463520576200_0001_1
2016-05-17 21:29:36,389 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520576200_0001_00_000001
2016-05-17 21:29:36,390 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520576200_0001_1_00_000000_0] to container: [container_1463520576200_0001_00_000001]
2016-05-17 21:29:36,390 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576200_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:36,390 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520576200_0001_00_000001 given task: attempt_1463520576200_0001_1_00_000000_0
2016-05-17 21:29:36,390 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520576200_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520576200_0001_1_00_000000_0
2016-05-17 21:29:36,390 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520576200_0001_1_00_000000_0] started. Is using containerId: [container_1463520576200_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:36,390 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B36A1AE2CD3E4E22B0605AA7275E1431, taskAttemptId=attempt_1463520576200_0001_1_00_000000_0, startTime=1463520576390, containerId=container_1463520576200_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520576200_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:36 Starting to run new task attempt: attempt_1463520576200_0001_1_00_000000_0
2016-05-17 21:29:36,391 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520576200_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:36,391 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:36,391 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520576200_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:36,391 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520576200_0001_1_00_000000_0
2016-05-17 21:29:36,391 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:36,391 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:36,391 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:36,391 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:36,391 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:36,391 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:36,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2DF36936CD2A4DEDA003185B6AEE8035/BE18C468009F46F1989AF30C8D911A5B] every/(1/1) ...latform/0_'field'_'field', VertexName: B36A1AE2CD3E4E22B0605AA7275E1431, VertexParallelism: 1, TaskAttemptID:attempt_1463520576200_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A074FC18D84B49569DB0E847D37BB0D8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A69B866D1BC247D5922FF938591F777C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:36,392 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:36,392 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520576200_0001_1_00_000000_0
2016-05-17 21:29:36,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:36,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:36,393 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520576200_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:36,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A69B866D1BC247D5922FF938591F777C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:36,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:36,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A074FC18D84B49569DB0E847D37BB0D8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:36,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:36,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A074FC18D84B49569DB0E847D37BB0D8
2016-05-17 21:29:36,395 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A074FC18D84B49569DB0E847D37BB0D8
2016-05-17 21:29:36,396 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:36,397 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:36,397 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A074FC18D84B49569DB0E847D37BB0D8
2016-05-17 21:29:36,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A074FC18D84B49569DB0E847D37BB0D8
2016-05-17 21:29:36,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:36,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A69B866D1BC247D5922FF938591F777C
2016-05-17 21:29:36,400 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:36,400 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A69B866D1BC247D5922FF938591F777C
2016-05-17 21:29:36,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A69B866D1BC247D5922FF938591F777C
2016-05-17 21:29:36,401 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B36A1AE2CD3E4E22B0605AA7275E1431, ordinal: 0
2016-05-17 21:29:36,401 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B36A1AE2CD3E4E22B0605AA7275E1431, mem on start (mb), free: 248, total: 358, max: 672
2016-05-17 21:29:36,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:36,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:36,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:36,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:36,402 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:36,402 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:36,402 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:36,402 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:36,402 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:36,402 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:36,402 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A074FC18D84B49569DB0E847D37BB0D8, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:36,403 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A69B866D1BC247D5922FF938591F777C, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:36,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:36,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A074FC18D84B49569DB0E847D37BB0D8 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:36,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A074FC18D84B49569DB0E847D37BB0D8
2016-05-17 21:29:36,403 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:36,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A074FC18D84B49569DB0E847D37BB0D8
2016-05-17 21:29:36,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:36,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:36,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:36,404 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520576200_0001_1_00_000000_0
2016-05-17 21:29:36,405 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:36,406 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:36,406 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3b8eec02
2016-05-17 21:29:36,406 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: A074FC18D84B49569DB0E847D37BB0D8
2016-05-17 21:29:36,406 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 6789C25E03624D619CD67AB7A53CD764
2016-05-17 21:29:36,407 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 6789C25E03624D619CD67AB7A53CD764
2016-05-17 21:29:36,407 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3b8eec02
2016-05-17 21:29:36,407 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:36,407 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:36,408 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:36,410 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:36,410 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@325786cc; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@1052d4e6; comparator=cascading.tuple.hadoop.util.TupleComparator@7ca03522; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:36,410 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:36,410 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:36,411 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:36,426 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:36,426 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:36,426 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:36,426 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:36,427 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:36,427 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:36,427 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:36,427 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B36A1AE2CD3E4E22B0605AA7275E1431, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:36,430 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520576200_0001_1_00_000000_0
2016-05-17 21:29:36,430 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B36A1AE2CD3E4E22B0605AA7275E1431, mem on close (mb), free: 146, total: 358, max: 672
2016-05-17 21:29:36,430 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:36,430 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:36,430 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:36,430 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:36,434 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:36 Completed running task attempt: attempt_1463520576200_0001_1_00_000000_0
2016-05-17 21:29:36,437 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520576200_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:36,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520576427, Process_Duration=3, Process_End_Time=1463520576430, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:36,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520576200_0001_1_00, eventCount=1
2016-05-17 21:29:36,438 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:36,438 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:36,438 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520576200_0001_00_000001
2016-05-17 21:29:36,438 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B36A1AE2CD3E4E22B0605AA7275E1431, taskAttemptId=attempt_1463520576200_0001_1_00_000000_0, startTime=1463520576390, finishTime=1463520576438, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520576427, Process_Duration=3, Process_End_Time=1463520576430, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:36,438 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:36,439 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520576200_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:36,439 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520576200_0001_1_00_000000_0
2016-05-17 21:29:36,440 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:36,441 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:TASK_FINISHED]: vertexName=B36A1AE2CD3E4E22B0605AA7275E1431, taskId=task_1463520576200_0001_1_00_000000, startTime=1463520576390, finishTime=1463520576439, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1463520576200_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520576427, Process_Duration=3, Process_End_Time=1463520576430, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:36,441 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520576200_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:36,441 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431] : 1
2016-05-17 21:29:36,441 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431]
2016-05-17 21:29:36,442 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:VERTEX_FINISHED]: vertexName=B36A1AE2CD3E4E22B0605AA7275E1431, vertexId=vertex_1463520576200_0001_1_00, initRequestedTime=1463520576367, initedTime=1463520576380, startRequestedTime=1463520576380, startedTime=1463520576380, finishTime=1463520576441, timeTaken=61, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520576427, Process_Duration=3, Process_End_Time=1463520576430, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520576390, firstTasksToStart=[ task_1463520576200_0001_1_00_000000 ], lastTaskFinishTime=1463520576438, lastTasksToFinish=[ task_1463520576200_0001_1_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520576200_0001_1_00_000000 ], longestDurationTasks=[ task_1463520576200_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:36,442 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:36,442 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576200_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:36,442 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C] attempt: attempt_1463520576200_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:36,443 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A69B866D1BC247D5922FF938591F777C
2016-05-17 21:29:36,443 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C]
2016-05-17 21:29:36,443 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:36,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:TASK_STARTED]: vertexName=A69B866D1BC247D5922FF938591F777C, taskId=task_1463520576200_0001_1_01_000000, scheduledTime=1463520576443, launchTime=1463520576443
2016-05-17 21:29:36,443 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520576200_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:36,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:36,444 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C]
2016-05-17 21:29:36,444 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:36,445 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520576200_0001_00_000001, stoppedTime=1463520576445, exitStatus=0
2016-05-17 21:29:36,445 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576200_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:36,445 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2DF36936CD2A4DEDA003185B6AEE8035/BE18C468009F46F1989AF30C8D911A5B] every/(1/1) ...latform/0_'field'_'field', VertexName: A69B866D1BC247D5922FF938591F777C, VertexParallelism: 1, TaskAttemptID:attempt_1463520576200_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B36A1AE2CD3E4E22B0605AA7275E1431, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=05ACBD6B31BD4DEC9940682EA834E7B7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:36,445 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:36,445 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:36,445 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520576200_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:36,446 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520576200_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:36,446 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:36,446 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576200_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:36,447 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520576200_0001_00_000002
2016-05-17 21:29:36,447 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576200_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:36,448 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520576200_0001_00_000002
2016-05-17 21:29:36,448 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:36,449 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520576200_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:36,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520576200_0001_00_000002, launchTime=1463520576449
2016-05-17 21:29:36,449 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576200_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:36,449 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:36,450 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520576200_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:36,451 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520576200_0001_00_000001, Exiting
2016-05-17 21:29:36,451 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520576200_0001_00_000001
2016-05-17 21:29:36,451 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520576200_0001_00_000002
2016-05-17 21:29:36,451 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520576200_0001_1_01_000000_0] to container: [container_1463520576200_0001_00_000002]
2016-05-17 21:29:36,452 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576200_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:36,452 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520576200_0001_00_000002 given task: attempt_1463520576200_0001_1_01_000000_0
2016-05-17 21:29:36,451 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520576200_0001_00_000001
2016-05-17 21:29:36,452 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520576200_0001_1_01_000000_0] started. Is using containerId: [container_1463520576200_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:36,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:36,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A69B866D1BC247D5922FF938591F777C, taskAttemptId=attempt_1463520576200_0001_1_01_000000_0, startTime=1463520576452, containerId=container_1463520576200_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520576200_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:36,453 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520576200_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:36,454 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520576200_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:36,454 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520576200_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520576200_0001_1_01_000000_0
2016-05-17 21:29:36 Starting to run new task attempt: attempt_1463520576200_0001_1_01_000000_0
2016-05-17 21:29:36,454 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520576200_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:36,454 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520576200_0001_1_01_000000_0
2016-05-17 21:29:36,455 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:36,455 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520576200_0001_00_000001 completed successfully
2016-05-17 21:29:36,455 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:36,455 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576200_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:36,455 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:36,456 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:36,456 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:36,456 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:36,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2DF36936CD2A4DEDA003185B6AEE8035/BE18C468009F46F1989AF30C8D911A5B] every/(1/1) ...latform/0_'field'_'field', VertexName: A69B866D1BC247D5922FF938591F777C, VertexParallelism: 1, TaskAttemptID:attempt_1463520576200_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B36A1AE2CD3E4E22B0605AA7275E1431, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=05ACBD6B31BD4DEC9940682EA834E7B7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:36,456 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:36,458 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520576200_0001_1_01_000000_0
2016-05-17 21:29:36,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:36,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:36,458 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520576200_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:36,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:36,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=05ACBD6B31BD4DEC9940682EA834E7B7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:36,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:36,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 05ACBD6B31BD4DEC9940682EA834E7B7
2016-05-17 21:29:36,461 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:36,461 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:36,461 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 05ACBD6B31BD4DEC9940682EA834E7B7
2016-05-17 21:29:36,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B36A1AE2CD3E4E22B0605AA7275E1431, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:36,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:36,466 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B36A1AE2CD3E4E22B0605AA7275E1431
2016-05-17 21:29:36,467 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:36,467 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:36,468 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B36A1AE2CD3E4E22B0605AA7275E1431
2016-05-17 21:29:36,468 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B36A1AE2CD3E4E22B0605AA7275E1431
2016-05-17 21:29:36,469 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A69B866D1BC247D5922FF938591F777C, ordinal: 1
2016-05-17 21:29:36,469 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A69B866D1BC247D5922FF938591F777C, mem on start (mb), free: 139, total: 358, max: 672
2016-05-17 21:29:36,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:36,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:36,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:36,471 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:36,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 05ACBD6B31BD4DEC9940682EA834E7B7
2016-05-17 21:29:36,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:36,471 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:36,471 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:36,471 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:36,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:36,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:36,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:36,472 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 05ACBD6B31BD4DEC9940682EA834E7B7, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:36,472 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B36A1AE2CD3E4E22B0605AA7275E1431, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:36,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:36,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B36A1AE2CD3E4E22B0605AA7275E1431 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:36,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B36A1AE2CD3E4E22B0605AA7275E1431
2016-05-17 21:29:36,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:36,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:36,473 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:36,473 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:36,473 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B36A1AE2CD3E4E22B0605AA7275E1431 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:36,474 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:36,474 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:36,474 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:36,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B36A1AE2CD3E4E22B0605AA7275E1431
2016-05-17 21:29:36,475 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:36,476 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520576200_0001_1_01_000000_0
2016-05-17 21:29:36,477 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520576200_0001_1_00_000000_0_10123, runDuration: 37782, ]
2016-05-17 21:29:36,480 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:36,480 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520576200_0001_1_00_000000_0_10123] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520576480, TimeTaken=3, Rate=0.06 MB/s
2016-05-17 21:29:36,481 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B36A1AE2CD3E4E22B0605AA7275E1431
2016-05-17 21:29:36,480 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 6789C25E03624D619CD67AB7A53CD764
2016-05-17 21:29:36,481 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_'field'"], id: 05ACBD6B31BD4DEC9940682EA834E7B7
2016-05-17 21:29:36,481 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_'field'"]
2016-05-17 21:29:36,481 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 6789C25E03624D619CD67AB7A53CD764, for 1 inputs
2016-05-17 21:29:36,481 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:36,482 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B36A1AE2CD3E4E22B0605AA7275E1431] #21 in 5ms
2016-05-17 21:29:36,482 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B36A1AE2CD3E4E22B0605AA7275E1431] #21, status:false, isInterrupted:false
2016-05-17 21:29:36,483 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:36,484 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:36,484 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:36,484 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:36,484 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:36,485 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:36,485 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B36A1AE2CD3E4E22B0605AA7275E1431
2016-05-17 21:29:36,485 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:36,485 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A69B866D1BC247D5922FF938591F777C, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:36,488 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:36,490 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@5395fbac; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@1bbe0413
2016-05-17 21:29:36,493 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:36,493 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:36,493 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520576200_0001_1_01_000000_0
2016-05-17 21:29:36,493 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520576200_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:36,494 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520576200_0001_1_01_000000_0
2016-05-17 21:29:36,494 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A69B866D1BC247D5922FF938591F777C, mem on close (mb), free: 134, total: 358, max: 672
2016-05-17 21:29:36,494 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B36A1AE2CD3E4E22B0605AA7275E1431
2016-05-17 21:29:36,495 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:36,496 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:36,496 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:36,496 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:36,496 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:36,496 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:36 Completed running task attempt: attempt_1463520576200_0001_1_01_000000_0
2016-05-17 21:29:36,497 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520576200_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:36,499 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520576485, Process_Duration=9, Process_End_Time=1463520576494, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:36,499 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A69B866D1BC247D5922FF938591F777C, taskAttemptId=attempt_1463520576200_0001_1_01_000000_0, startTime=1463520576452, finishTime=1463520576499, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520576485, Process_Duration=9, Process_End_Time=1463520576494, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:36,499 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:36,500 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:36,499 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:36,499 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520576200_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:36,500 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520576200_0001_1_01_000000_0
2016-05-17 21:29:36,500 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:36,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:TASK_FINISHED]: vertexName=A69B866D1BC247D5922FF938591F777C, taskId=task_1463520576200_0001_1_01_000000, startTime=1463520576452, finishTime=1463520576500, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1463520576200_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520576485, Process_Duration=9, Process_End_Time=1463520576494, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:36,501 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520576200_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:36,501 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C] : 1
2016-05-17 21:29:36,501 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C]
2016-05-17 21:29:36,501 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:36,501 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520576200_0001_00_000002
2016-05-17 21:29:36,502 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:VERTEX_FINISHED]: vertexName=A69B866D1BC247D5922FF938591F777C, vertexId=vertex_1463520576200_0001_1_01, initRequestedTime=1463520576370, initedTime=1463520576378, startRequestedTime=1463520576383, startedTime=1463520576383, finishTime=1463520576501, timeTaken=118, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375390208, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520576485, Process_Duration=9, Process_End_Time=1463520576494, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520576452, firstTasksToStart=[ task_1463520576200_0001_1_01_000000 ], lastTaskFinishTime=1463520576499, lastTasksToFinish=[ task_1463520576200_0001_1_01_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520576200_0001_1_01_000000 ], longestDurationTasks=[ task_1463520576200_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:36,502 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:36,502 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576200_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:36,502 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520576200_0001_00_000002, stoppedTime=1463520576502, exitStatus=0
2016-05-17 21:29:36,502 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576200_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:36,502 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:36,503 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520576200_0001_1
2016-05-17 21:29:36,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520576200_0001_1
2016-05-17 21:29:36,503 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520576200_0001_1_00 [B36A1AE2CD3E4E22B0605AA7275E1431]
2016-05-17 21:29:36,503 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 05ACBD6B31BD4DEC9940682EA834E7B7 for vertex: vertex_1463520576200_0001_1_01 [A69B866D1BC247D5922FF938591F777C]
2016-05-17 21:29:36,503 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:36,504 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:36,513 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520576200_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:36,513 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520576200_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:36,513 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520576200_0001_00_000002, Exiting
2016-05-17 21:29:36,513 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520576200_0001_00_000002
2016-05-17 21:29:36,513 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520576200_0001_00_000002
2016-05-17 21:29:36,514 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520576200_0001_1, queueSize=0
2016-05-17 21:29:36,514 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:36,514 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576200_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520576200_0001_1, startTime=1463520576367, finishTime=1463520576512, timeTaken=145, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=345, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=750780416, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041152912, Process_Duration=12, Process_End_Time=2927041152924, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:36,514 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520576200_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:36,514 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520576200_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:36,514 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520576200_0001_00_000002
2016-05-17 21:29:36,514 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520576200_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:36,515 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520576200_0001_00_000002 completed successfully
2016-05-17 21:29:36 Completed Dag: dag_1463520576200_0001_1
2016-05-17 21:29:36,515 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520576200_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:36,515 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520576200_0001_1_post
2016-05-17 21:29:36,515 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:36,515 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:36,516 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576200_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:36,516 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576200_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:36,521 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_'field'_'field', applicationId=application_1463520576200_0001
2016-05-17 21:29:36,523 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:36,523 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:36,523 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:36,523 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:36,523 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:36,524 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:36,524 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:36,524 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:36,524 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:36,526 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:36,526 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:36,526 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:36,526 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:36,533 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2DF36936CD2A4DEDA003185B6AEE8035/BE18C468009F46F1989AF30C8D911A5B/.tez/application_1463520576200_0001
2016-05-17 21:29:36,534 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_'field'/_temporary
2016-05-17 21:29:36,534 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.352
2016-05-17 21:29:36,608 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_'field'
2016-05-17 21:29:36,635 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:36,635 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:36,667 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 21:29:36,674 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 21:29:36,675 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:36,675 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:36,675 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:36,695 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:36,703 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_RESULTS
2016-05-17 21:29:36,753 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:36,753 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:36,768 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:36,768 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:36,768 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_RESULTS"]
2016-05-17 21:29:36,768 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:36,769 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:36,769 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:36,770 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...latform/0_'field'_RESULTS
2016-05-17 21:29:36,770 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:36,788 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:36,794 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:36,794 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:36,794 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:36,806 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/ED9BABA3FF02429A9586878EE67E1950/A6B1E5B4ACD943C38A702AF25DF57CDE/.tez/application_1463520576794_0001 doesn't exist and is created
2016-05-17 21:29:36,822 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:36,823 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:36,823 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/ED9BABA3FF02429A9586878EE67E1950/A6B1E5B4ACD943C38A702AF25DF57CDE/.tez/application_1463520576794_0001
2016-05-17 21:29:36,825 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520576794_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:36,836 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:36,836 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:36,838 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:36,839 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:36,839 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/ED9BABA3FF02429A9586878EE67E1950/A6B1E5B4ACD943C38A702AF25DF57CDE/.tez/application_1463520576794_0001/localmode-log-dir
2016-05-17 21:29:36,839 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/ED9BABA3FF02429A9586878EE67E1950/A6B1E5B4ACD943C38A702AF25DF57CDE/.tez/application_1463520576794_0001/localmode-log-dir/history.txt.appattempt_1463520576794_0001_000000, maxErrors=10
2016-05-17 21:29:36,839 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:36,840 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520576794_0001_000000, appSubmitTime=1463520576825, launchTime=1463520576825
2016-05-17 21:29:36,843 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:36,846 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:36,848 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56717
2016-05-17 21:29:36,848 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:36,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:36,851 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:36,853 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520576794_0001_000000, startTime=1463520576852
2016-05-17 21:29:36,853 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:36,853 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:36,923 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:36,923 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:36,923 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_'field'_RESULTS, applicationId=application_1463520576794_0001, dagName=[ED9BABA3FF02429A9586878EE67E1950/A6B1E5B4ACD943C38A702AF25DF57CDE] every/(1/1) ...latform/0_'field'_RESULTS
2016-05-17 21:29:36,938 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [ED9BABA3FF02429A9586878EE67E1950/A6B1E5B4ACD943C38A702AF25DF57CDE] every/(1/1) ...latform/0_'field'_RESULTS
2016-05-17 21:29:36,939 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520576794_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/ED9BABA3FF02429A9586878EE67E1950/A6B1E5B4ACD943C38A702AF25DF57CDE/.tez/application_1463520576794_0001/localmode-log-dir/dag_1463520576794_0001_1.dot
2016-05-17 21:29:36,940 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/ED9BABA3FF02429A9586878EE67E1950/A6B1E5B4ACD943C38A702AF25DF57CDE/.tez/application_1463520576794_0001/localmode-log-dir/dag_1463520576794_0001_1-tez-dag.pb.txt
2016-05-17 21:29:36,945 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520576794_0001_1
2016-05-17 21:29:36,945 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:36,945 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [ED9BABA3FF02429A9586878EE67E1950/A6B1E5B4ACD943C38A702AF25DF57CDE] every/(1/1) ...latform/0_'field'_RESULTS
2016-05-17 21:29:36 Running Dag: dag_1463520576794_0001_1
2016-05-17 21:29:36,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520576794_0001_1, submitTime=1463520576938
2016-05-17 21:29:36,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:36,956 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D52909F85B8540AF965E8ED255F48174
2016-05-17 21:29:36,956 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:36,957 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:36,957 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520576794_0001_1, initTime=1463520576956
2016-05-17 21:29:36,957 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520576794_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:36,957 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:DAG_STARTED]: dagID=dag_1463520576794_0001_1, startTime=1463520576957
2016-05-17 21:29:36,957 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:36,957 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:36,958 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520576794_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:36,957 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_'field'_RESULTS, applicationId=application_1463520576794_0001, dagName=[ED9BABA3FF02429A9586878EE67E1950/A6B1E5B4ACD943C38A702AF25DF57CDE] every/(1/1) ...latform/0_'field'_RESULTS
2016-05-17 21:29:36,958 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D41A7E0626E248EDB6953CE7F6418FBB : {A32E4D906DED4FCD9B3D034B1A8C90FB={InputName=A32E4D906DED4FCD9B3D034B1A8C90FB}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:36,958 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A32E4D906DED4FCD9B3D034B1A8C90FB, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:36,958 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB]
2016-05-17 21:29:36,958 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB]
2016-05-17 21:29:36,958 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB]
2016-05-17 21:29:36,959 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB]. Starting root input initializers: 1
2016-05-17 21:29:36,959 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:36,959 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174]
2016-05-17 21:29:36,960 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A32E4D906DED4FCD9B3D034B1A8C90FB on vertex vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB]
2016-05-17 21:29:36,962 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:36,962 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174]
2016-05-17 21:29:36,962 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174]
2016-05-17 21:29:36,963 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174]
2016-05-17 21:29:36,963 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=42B06F9B1185432680D508FEE120D0EC, vertexId=vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:36,963 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:36,963 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=42B06F9B1185432680D508FEE120D0EC, vertexId=vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174]
2016-05-17 21:29:36,963 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A32E4D906DED4FCD9B3D034B1A8C90FB asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:36,964 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:36,964 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:36,965 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=42B06F9B1185432680D508FEE120D0EC, vertexName=D52909F85B8540AF965E8ED255F48174, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:36,965 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=42B06F9B1185432680D508FEE120D0EC, vertexId=vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174]
2016-05-17 21:29:36,966 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:36,966 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A32E4D906DED4FCD9B3D034B1A8C90FB on vertex vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB]
2016-05-17 21:29:36,967 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:36,967 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D52909F85B8540AF965E8ED255F48174, vertexId=vertex_1463520576794_0001_1_01, initRequestedTime=1463520576959, initedTime=1463520576967, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:36,967 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:36,967 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:36,968 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A32E4D906DED4FCD9B3D034B1A8C90FB=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:36,968 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB] parallelism set to 1
2016-05-17 21:29:36,968 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:36,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520576794_0001_1_00, eventCount=1
2016-05-17 21:29:36,969 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:36,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D41A7E0626E248EDB6953CE7F6418FBB, vertexId=vertex_1463520576794_0001_1_00, initRequestedTime=1463520576958, initedTime=1463520576969, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:36,970 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB] with distanceFromRoot: 0
2016-05-17 21:29:36,970 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:36,970 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D41A7E0626E248EDB6953CE7F6418FBB
2016-05-17 21:29:36,970 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB]
2016-05-17 21:29:36,971 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:36,971 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520576794_0001_1_00, startRequestedTime=1463520576970, startedTime=1463520576970
2016-05-17 21:29:36,971 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:36,972 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:36,972 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:TASK_STARTED]: vertexName=D41A7E0626E248EDB6953CE7F6418FBB, taskId=task_1463520576794_0001_1_00_000000, scheduledTime=1463520576971, launchTime=1463520576971
2016-05-17 21:29:36,972 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520576794_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:36,972 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520576794_0001_1_00 for vertex: vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174] numStartedSources: 1 numSources: 1
2016-05-17 21:29:36,973 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174] with distanceFromRoot: 1
2016-05-17 21:29:36,973 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB]
2016-05-17 21:29:36,973 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D41A7E0626E248EDB6953CE7F6418FBB in vertex: D52909F85B8540AF965E8ED255F48174
2016-05-17 21:29:36,973 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D52909F85B8540AF965E8ED255F48174 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:36,974 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:36,974 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520576794_0001_1_01, startRequestedTime=1463520576973, startedTime=1463520576973
2016-05-17 21:29:36,974 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:36,974 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [ED9BABA3FF02429A9586878EE67E1950/A6B1E5B4ACD943C38A702AF25DF57CDE] every/(1/1) ...latform/0_'field'_RESULTS, VertexName: D41A7E0626E248EDB6953CE7F6418FBB, VertexParallelism: 1, TaskAttemptID:attempt_1463520576794_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A32E4D906DED4FCD9B3D034B1A8C90FB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D52909F85B8540AF965E8ED255F48174, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:36,975 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520576794_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:36,975 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:36,975 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:36,976 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:36,976 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:36,976 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520576794_0001_00_000001
2016-05-17 21:29:36,977 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576794_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:36,978 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520576794_0001_00_000001
2016-05-17 21:29:36,978 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:36,978 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576794_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:36,978 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:36,979 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520576794_0001_00_000001, launchTime=1463520576978
2016-05-17 21:29:36,979 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520576794_0001_00_000001
2016-05-17 21:29:36,980 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520576794_0001_1_00_000000_0] to container: [container_1463520576794_0001_00_000001]
2016-05-17 21:29:36,980 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576794_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:36,980 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520576794_0001_1_00_000000_0] started. Is using containerId: [container_1463520576794_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:36,981 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D41A7E0626E248EDB6953CE7F6418FBB, taskAttemptId=attempt_1463520576794_0001_1_00_000000_0, startTime=1463520576980, containerId=container_1463520576794_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520576794_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:36,981 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520576794_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:36,981 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520576794_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:36,981 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:36,981 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520576794_0001_00_000001 given task: attempt_1463520576794_0001_1_00_000000_0
2016-05-17 21:29:36,981 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520576794_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520576794_0001_1_00_000000_0
2016-05-17 21:29:36 Starting to run new task attempt: attempt_1463520576794_0001_1_00_000000_0
2016-05-17 21:29:36,982 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520576794_0001_1_00_000000_0
2016-05-17 21:29:36,982 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:36,982 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:36,982 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:36,982 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:36,982 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:36,983 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:36,983 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [ED9BABA3FF02429A9586878EE67E1950/A6B1E5B4ACD943C38A702AF25DF57CDE] every/(1/1) ...latform/0_'field'_RESULTS, VertexName: D41A7E0626E248EDB6953CE7F6418FBB, VertexParallelism: 1, TaskAttemptID:attempt_1463520576794_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A32E4D906DED4FCD9B3D034B1A8C90FB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D52909F85B8540AF965E8ED255F48174, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:36,983 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:36,983 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520576794_0001, with dag id: dag_1463520576794_0001_1
2016-05-17 21:29:36,984 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520576794_0001_1_00_000000_0
2016-05-17 21:29:36,984 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:36,984 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:36,984 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520576794_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:36,986 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A32E4D906DED4FCD9B3D034B1A8C90FB, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:36,986 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:36,987 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:36,987 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A32E4D906DED4FCD9B3D034B1A8C90FB
2016-05-17 21:29:36,987 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A32E4D906DED4FCD9B3D034B1A8C90FB
2016-05-17 21:29:36,988 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:36,989 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:36,989 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A32E4D906DED4FCD9B3D034B1A8C90FB
2016-05-17 21:29:36,989 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A32E4D906DED4FCD9B3D034B1A8C90FB
2016-05-17 21:29:36,990 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D52909F85B8540AF965E8ED255F48174, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:36,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:36,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D52909F85B8540AF965E8ED255F48174
2016-05-17 21:29:36,994 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:36,994 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D52909F85B8540AF965E8ED255F48174
2016-05-17 21:29:36,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D52909F85B8540AF965E8ED255F48174
2016-05-17 21:29:36,996 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D41A7E0626E248EDB6953CE7F6418FBB, ordinal: 0
2016-05-17 21:29:36,996 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D41A7E0626E248EDB6953CE7F6418FBB, mem on start (mb), free: 243, total: 358, max: 672
2016-05-17 21:29:36,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:36,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:36,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:36,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:36,997 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:36,997 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:36,997 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:36,997 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:36,997 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:36,997 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:36,998 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A32E4D906DED4FCD9B3D034B1A8C90FB, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:36,998 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D52909F85B8540AF965E8ED255F48174, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:36,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:36,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A32E4D906DED4FCD9B3D034B1A8C90FB
2016-05-17 21:29:36,998 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:36,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A32E4D906DED4FCD9B3D034B1A8C90FB
2016-05-17 21:29:36,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A32E4D906DED4FCD9B3D034B1A8C90FB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:36,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:36,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:36,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:36,999 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520576794_0001_1_00_000000_0
2016-05-17 21:29:37,004 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:37,005 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@39e7cd0a
2016-05-17 21:29:37,009 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:37,010 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: A32E4D906DED4FCD9B3D034B1A8C90FB
2016-05-17 21:29:37,010 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 6FF6A67E5AFF437CA7D83E7915812D49
2016-05-17 21:29:37,010 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 6FF6A67E5AFF437CA7D83E7915812D49
2016-05-17 21:29:37,010 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@39e7cd0a
2016-05-17 21:29:37,011 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:37,011 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:37,012 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:37,014 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:37,014 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@77424e9; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@5a2340d1; comparator=cascading.tuple.hadoop.util.TupleComparator@40748792; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:37,014 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:37,014 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:37,014 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:37,030 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:37,030 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:37,030 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:37,030 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:37,030 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:37,031 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:37,031 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:37,031 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D41A7E0626E248EDB6953CE7F6418FBB, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:37,034 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520576794_0001_1_00_000000_0
2016-05-17 21:29:37,034 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D41A7E0626E248EDB6953CE7F6418FBB, mem on close (mb), free: 141, total: 358, max: 672
2016-05-17 21:29:37,034 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:37,034 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:37,034 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:37,034 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:37,039 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:37 Completed running task attempt: attempt_1463520576794_0001_1_00_000000_0
2016-05-17 21:29:37,042 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520576794_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:37,042 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520576794_0001_1_00, eventCount=1
2016-05-17 21:29:37,043 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:37,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D41A7E0626E248EDB6953CE7F6418FBB, taskAttemptId=attempt_1463520576794_0001_1_00_000000_0, startTime=1463520576980, finishTime=1463520577043, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520577031, Process_Duration=3, Process_End_Time=1463520577034, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:37,043 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520576794_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:37,043 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:37,043 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520576794_0001_1_00_000000_0
2016-05-17 21:29:37,044 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:37,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:TASK_FINISHED]: vertexName=D41A7E0626E248EDB6953CE7F6418FBB, taskId=task_1463520576794_0001_1_00_000000, startTime=1463520576980, finishTime=1463520577043, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1463520576794_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520577031, Process_Duration=3, Process_End_Time=1463520577034, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:37,044 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520576794_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:37,044 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB] : 1
2016-05-17 21:29:37,044 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB]
2016-05-17 21:29:37,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:37,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:VERTEX_FINISHED]: vertexName=D41A7E0626E248EDB6953CE7F6418FBB, vertexId=vertex_1463520576794_0001_1_00, initRequestedTime=1463520576958, initedTime=1463520576969, startRequestedTime=1463520576970, startedTime=1463520576970, finishTime=1463520577044, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520577031, Process_Duration=3, Process_End_Time=1463520577034, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520576980, firstTasksToStart=[ task_1463520576794_0001_1_00_000000 ], lastTaskFinishTime=1463520577043, lastTasksToFinish=[ task_1463520576794_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520576794_0001_1_00_000000 ], longestDurationTasks=[ task_1463520576794_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:37,046 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:37,046 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576794_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:37,046 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174] attempt: attempt_1463520576794_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:37,046 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D52909F85B8540AF965E8ED255F48174
2016-05-17 21:29:37,046 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174]
2016-05-17 21:29:37,047 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520576794_0001_00_000001, stoppedTime=1463520577046, exitStatus=0
2016-05-17 21:29:37,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:37,047 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576794_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:37,047 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:37,047 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:TASK_STARTED]: vertexName=D52909F85B8540AF965E8ED255F48174, taskId=task_1463520576794_0001_1_01_000000, scheduledTime=1463520577047, launchTime=1463520577047
2016-05-17 21:29:37,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:37,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520577031, Process_Duration=3, Process_End_Time=1463520577034, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:37,047 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520576794_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:37,047 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174]
2016-05-17 21:29:37,047 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520576794_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:37,048 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [ED9BABA3FF02429A9586878EE67E1950/A6B1E5B4ACD943C38A702AF25DF57CDE] every/(1/1) ...latform/0_'field'_RESULTS, VertexName: D52909F85B8540AF965E8ED255F48174, VertexParallelism: 1, TaskAttemptID:attempt_1463520576794_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D41A7E0626E248EDB6953CE7F6418FBB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=42B06F9B1185432680D508FEE120D0EC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:37,048 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520576794_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:37,048 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576794_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:37,048 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:37,049 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:37,049 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:37,050 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520576794_0001_00_000002
2016-05-17 21:29:37,051 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576794_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:37,051 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520576794_0001_00_000001
2016-05-17 21:29:37,051 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520576794_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:37,051 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520576794_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:37,052 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520576794_0001_00_000001, Exiting
2016-05-17 21:29:37,052 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520576794_0001_00_000001
2016-05-17 21:29:37,052 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520576794_0001_00_000001
2016-05-17 21:29:37,052 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520576794_0001_00_000002
2016-05-17 21:29:37,053 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520576794_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:37,053 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:37,053 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520576794_0001_00_000001 completed successfully
2016-05-17 21:29:37,055 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576794_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:37,055 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576794_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:37,056 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:37,056 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520576794_0001_00_000002, launchTime=1463520577056
2016-05-17 21:29:37,056 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520576794_0001_00_000002
2016-05-17 21:29:37,057 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520576794_0001_1_01_000000_0] to container: [container_1463520576794_0001_00_000002]
2016-05-17 21:29:37,057 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576794_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:37,057 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520576794_0001_00_000002 given task: attempt_1463520576794_0001_1_01_000000_0
2016-05-17 21:29:37,057 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520576794_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520576794_0001_1_01_000000_0
2016-05-17 21:29:37,057 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520576794_0001_1_01_000000_0] started. Is using containerId: [container_1463520576794_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:37,058 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D52909F85B8540AF965E8ED255F48174, taskAttemptId=attempt_1463520576794_0001_1_01_000000_0, startTime=1463520577057, containerId=container_1463520576794_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520576794_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:37,058 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:37,058 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520576794_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:37,058 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520576794_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:37 Starting to run new task attempt: attempt_1463520576794_0001_1_01_000000_0
2016-05-17 21:29:37,059 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520576794_0001_1_01_000000_0
2016-05-17 21:29:37,059 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:37,059 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:37,059 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:37,059 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:37,059 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:37,059 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:37,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [ED9BABA3FF02429A9586878EE67E1950/A6B1E5B4ACD943C38A702AF25DF57CDE] every/(1/1) ...latform/0_'field'_RESULTS, VertexName: D52909F85B8540AF965E8ED255F48174, VertexParallelism: 1, TaskAttemptID:attempt_1463520576794_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D41A7E0626E248EDB6953CE7F6418FBB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=42B06F9B1185432680D508FEE120D0EC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:37,060 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:37,060 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520576794_0001_1_01_000000_0
2016-05-17 21:29:37,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:37,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:37,060 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520576794_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:37,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D41A7E0626E248EDB6953CE7F6418FBB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:37,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:37,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=42B06F9B1185432680D508FEE120D0EC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:37,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:37,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 42B06F9B1185432680D508FEE120D0EC
2016-05-17 21:29:37,063 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:37,063 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 42B06F9B1185432680D508FEE120D0EC
2016-05-17 21:29:37,065 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:37,072 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:37,072 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D41A7E0626E248EDB6953CE7F6418FBB
2016-05-17 21:29:37,074 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D52909F85B8540AF965E8ED255F48174, ordinal: 1
2016-05-17 21:29:37,075 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D52909F85B8540AF965E8ED255F48174, mem on start (mb), free: 134, total: 358, max: 672
2016-05-17 21:29:37,075 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:37,075 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:37,076 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:37,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 42B06F9B1185432680D508FEE120D0EC
2016-05-17 21:29:37,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:37,076 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:37,076 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:37,077 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D41A7E0626E248EDB6953CE7F6418FBB
2016-05-17 21:29:37,077 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D41A7E0626E248EDB6953CE7F6418FBB
2016-05-17 21:29:37,077 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:37,077 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:37,077 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:37,078 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:37,078 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:37,078 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:37,079 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:37,079 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 42B06F9B1185432680D508FEE120D0EC, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:37,079 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D41A7E0626E248EDB6953CE7F6418FBB, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:37,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:37,080 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D41A7E0626E248EDB6953CE7F6418FBB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:37,080 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D41A7E0626E248EDB6953CE7F6418FBB
2016-05-17 21:29:37,080 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:37,081 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:37,081 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:37,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:37,081 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D41A7E0626E248EDB6953CE7F6418FBB with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:37,082 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:37,082 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:37,082 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:37,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D41A7E0626E248EDB6953CE7F6418FBB
2016-05-17 21:29:37,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:37,084 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520576794_0001_1_01_000000_0
2016-05-17 21:29:37,084 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520576794_0001_1_00_000000_0_10129, runDuration: 48746, ]
2016-05-17 21:29:37,085 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520576794_0001_1_00_000000_0_10129] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520577085, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:37,085 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D41A7E0626E248EDB6953CE7F6418FBB
2016-05-17 21:29:37,085 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:37,085 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D41A7E0626E248EDB6953CE7F6418FBB] #22 in 1ms
2016-05-17 21:29:37,085 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D41A7E0626E248EDB6953CE7F6418FBB] #22, status:false, isInterrupted:false
2016-05-17 21:29:37,086 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:37,087 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:37,087 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:37,087 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:37,087 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:37,087 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:37,087 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D41A7E0626E248EDB6953CE7F6418FBB
2016-05-17 21:29:37,087 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:37,089 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:37,090 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 6FF6A67E5AFF437CA7D83E7915812D49
2016-05-17 21:29:37,090 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_RESULTS"], id: 42B06F9B1185432680D508FEE120D0EC
2016-05-17 21:29:37,090 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_RESULTS"]
2016-05-17 21:29:37,090 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 6FF6A67E5AFF437CA7D83E7915812D49, for 1 inputs
2016-05-17 21:29:37,091 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D52909F85B8540AF965E8ED255F48174, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:37,092 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:37,094 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@1ffa895a; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@794f841c
2016-05-17 21:29:37,100 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:37,101 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:37,101 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520576794_0001_1_01_000000_0
2016-05-17 21:29:37,101 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520576794_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:37,102 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520576794_0001_1_01_000000_0
2016-05-17 21:29:37,102 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D52909F85B8540AF965E8ED255F48174, mem on close (mb), free: 129, total: 358, max: 672
2016-05-17 21:29:37,102 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D41A7E0626E248EDB6953CE7F6418FBB
2016-05-17 21:29:37,103 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:37,104 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:37,104 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:37,104 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:37,104 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:37,104 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:37 Completed running task attempt: attempt_1463520576794_0001_1_01_000000_0
2016-05-17 21:29:37,105 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520576794_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:37,106 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:37,106 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D52909F85B8540AF965E8ED255F48174, taskAttemptId=attempt_1463520576794_0001_1_01_000000_0, startTime=1463520577057, finishTime=1463520577106, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520577091, Process_Duration=11, Process_End_Time=1463520577102, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:37,106 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520576794_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:37,106 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520576794_0001_1_01_000000_0
2016-05-17 21:29:37,107 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:37,107 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:37,107 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:TASK_FINISHED]: vertexName=D52909F85B8540AF965E8ED255F48174, taskId=task_1463520576794_0001_1_01_000000, startTime=1463520577057, finishTime=1463520577106, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1463520576794_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520577091, Process_Duration=11, Process_End_Time=1463520577102, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:37,107 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520576794_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:37,107 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174] : 1
2016-05-17 21:29:37,107 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174]
2016-05-17 21:29:37,108 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:37,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:VERTEX_FINISHED]: vertexName=D52909F85B8540AF965E8ED255F48174, vertexId=vertex_1463520576794_0001_1_01, initRequestedTime=1463520576959, initedTime=1463520576967, startRequestedTime=1463520576973, startedTime=1463520576973, finishTime=1463520577107, timeTaken=134, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520577091, Process_Duration=11, Process_End_Time=1463520577102, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520577057, firstTasksToStart=[ task_1463520576794_0001_1_01_000000 ], lastTaskFinishTime=1463520577106, lastTasksToFinish=[ task_1463520576794_0001_1_01_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520576794_0001_1_01_000000 ], longestDurationTasks=[ task_1463520576794_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:37,109 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:37,109 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576794_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:37,109 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:37,109 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520576794_0001_00_000002, stoppedTime=1463520577109, exitStatus=0
2016-05-17 21:29:37,109 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576794_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:37,110 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:37,110 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520576794_0001_1
2016-05-17 21:29:37,110 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520576794_0001_1
2016-05-17 21:29:37,110 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520576794_0001_1_00 [D41A7E0626E248EDB6953CE7F6418FBB]
2016-05-17 21:29:37,110 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 42B06F9B1185432680D508FEE120D0EC for vertex: vertex_1463520576794_0001_1_01 [D52909F85B8540AF965E8ED255F48174]
2016-05-17 21:29:37,111 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520577091, Process_Duration=11, Process_End_Time=1463520577102, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:37,112 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:37,112 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:37,112 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520576794_0001_00_000002
2016-05-17 21:29:37,113 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520576794_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:37,113 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520576794_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:37,113 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520576794_0001_00_000002, Exiting
2016-05-17 21:29:37,113 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520576794_0001_00_000002
2016-05-17 21:29:37,113 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520576794_0001_00_000002
2016-05-17 21:29:37,114 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520576794_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:37,114 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520576794_0001_00_000002 completed successfully
2016-05-17 21:29:37,120 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520576794_0001_1, queueSize=0
2016-05-17 21:29:37,120 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:37,121 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520576794_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520576794_0001_1, startTime=1463520576957, finishTime=1463520577119, timeTaken=162, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=345, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041154122, Process_Duration=14, Process_End_Time=2927041154136, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:37,121 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520576794_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:37,121 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520576794_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:37,122 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520576794_0001_00_000002
2016-05-17 21:29:37,122 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520576794_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:37 Completed Dag: dag_1463520576794_0001_1
2016-05-17 21:29:37,123 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520576794_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:37,123 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520576794_0001_1_post
2016-05-17 21:29:37,123 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:37,123 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:37,129 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_'field'_RESULTS, applicationId=application_1463520576794_0001
2016-05-17 21:29:37,131 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:37,131 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:37,131 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:37,131 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:37,131 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:37,133 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:37,133 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:37,133 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:37,134 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:37,134 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:37,134 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:37,134 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:37,135 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:37,139 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/ED9BABA3FF02429A9586878EE67E1950/A6B1E5B4ACD943C38A702AF25DF57CDE/.tez/application_1463520576794_0001
2016-05-17 21:29:37,140 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_RESULTS/_temporary
2016-05-17 21:29:37,140 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.373
2016-05-17 21:29:37,216 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_RESULTS
2016-05-17 21:29:37,236 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:37,236 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:37,259 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:37,260 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:37,268 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:29:37,272 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 21:29:37,272 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:37,272 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:37,273 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:37,285 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:37,291 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_ALL
2016-05-17 21:29:37,331 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:37,332 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:37,332 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_ALL"]
2016-05-17 21:29:37,332 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:37,332 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:37,332 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:37,332 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...blyplatform/0_'field'_ALL
2016-05-17 21:29:37,334 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:37,345 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:37,350 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:37,350 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:37,350 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:37,359 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5CF2D6598A0D447AAC2ED7C0CE155DDA/F335E54EF5E84F65B2E1073D2648511A/.tez/application_1463520577350_0001 doesn't exist and is created
2016-05-17 21:29:37,373 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:37,373 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:37,374 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5CF2D6598A0D447AAC2ED7C0CE155DDA/F335E54EF5E84F65B2E1073D2648511A/.tez/application_1463520577350_0001
2016-05-17 21:29:37,376 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520577350_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:37,385 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:37,386 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:37,387 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:37,387 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:37,388 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5CF2D6598A0D447AAC2ED7C0CE155DDA/F335E54EF5E84F65B2E1073D2648511A/.tez/application_1463520577350_0001/localmode-log-dir
2016-05-17 21:29:37,388 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5CF2D6598A0D447AAC2ED7C0CE155DDA/F335E54EF5E84F65B2E1073D2648511A/.tez/application_1463520577350_0001/localmode-log-dir/history.txt.appattempt_1463520577350_0001_000000, maxErrors=10
2016-05-17 21:29:37,388 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:37,388 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520577350_0001_000000, appSubmitTime=1463520577376, launchTime=1463520577376
2016-05-17 21:29:37,390 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:37,391 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:37,392 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:37,393 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:37,394 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:47206
2016-05-17 21:29:37,396 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:37,398 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520577350_0001_000000, startTime=1463520577398
2016-05-17 21:29:37,398 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:37,398 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:37,474 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:37,474 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:37,474 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/0_'field'_ALL, applicationId=application_1463520577350_0001, dagName=[5CF2D6598A0D447AAC2ED7C0CE155DDA/F335E54EF5E84F65B2E1073D2648511A] every/(1/1) ...blyplatform/0_'field'_ALL
2016-05-17 21:29:37,488 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5CF2D6598A0D447AAC2ED7C0CE155DDA/F335E54EF5E84F65B2E1073D2648511A] every/(1/1) ...blyplatform/0_'field'_ALL
2016-05-17 21:29:37,489 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520577350_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5CF2D6598A0D447AAC2ED7C0CE155DDA/F335E54EF5E84F65B2E1073D2648511A/.tez/application_1463520577350_0001/localmode-log-dir/dag_1463520577350_0001_1.dot
2016-05-17 21:29:37,489 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5CF2D6598A0D447AAC2ED7C0CE155DDA/F335E54EF5E84F65B2E1073D2648511A/.tez/application_1463520577350_0001/localmode-log-dir/dag_1463520577350_0001_1-tez-dag.pb.txt
2016-05-17 21:29:37,496 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520577350_0001_1
2016-05-17 21:29:37,496 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:37,496 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5CF2D6598A0D447AAC2ED7C0CE155DDA/F335E54EF5E84F65B2E1073D2648511A] every/(1/1) ...blyplatform/0_'field'_ALL
2016-05-17 21:29:37 Running Dag: dag_1463520577350_0001_1
2016-05-17 21:29:37,507 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:37,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520577350_0001_1, submitTime=1463520577488
2016-05-17 21:29:37,508 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E4FE813CD0B94F5E9C78531CEE121BFB
2016-05-17 21:29:37,508 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:37,508 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:37,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520577350_0001_1, initTime=1463520577507
2016-05-17 21:29:37,509 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520577350_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:37,509 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:DAG_STARTED]: dagID=dag_1463520577350_0001_1, startTime=1463520577509
2016-05-17 21:29:37,509 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:37,509 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:37,510 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/0_'field'_ALL, applicationId=application_1463520577350_0001, dagName=[5CF2D6598A0D447AAC2ED7C0CE155DDA/F335E54EF5E84F65B2E1073D2648511A] every/(1/1) ...blyplatform/0_'field'_ALL
2016-05-17 21:29:37,510 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520577350_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:37,510 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FAD9703BC5A9450A9AE06EE9D076BDB0 : {D7B14FB1939947CEB4C1BB1626CD170E={InputName=D7B14FB1939947CEB4C1BB1626CD170E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:37,510 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: D7B14FB1939947CEB4C1BB1626CD170E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:37,510 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0]
2016-05-17 21:29:37,510 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0]
2016-05-17 21:29:37,511 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0]
2016-05-17 21:29:37,511 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0]. Starting root input initializers: 1
2016-05-17 21:29:37,512 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:37,512 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB]
2016-05-17 21:29:37,512 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: D7B14FB1939947CEB4C1BB1626CD170E on vertex vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0]
2016-05-17 21:29:37,514 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:37,514 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input D7B14FB1939947CEB4C1BB1626CD170E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:37,514 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:37,515 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:37,515 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:37,515 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB]
2016-05-17 21:29:37,515 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB]
2016-05-17 21:29:37,516 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB]
2016-05-17 21:29:37,516 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=C58D17288BCE4D37A73A28582D0C6C8F, vertexId=vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:37,516 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=C58D17288BCE4D37A73A28582D0C6C8F, vertexId=vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB]
2016-05-17 21:29:37,517 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:37,517 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: D7B14FB1939947CEB4C1BB1626CD170E on vertex vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0]
2016-05-17 21:29:37,518 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=C58D17288BCE4D37A73A28582D0C6C8F, vertexName=E4FE813CD0B94F5E9C78531CEE121BFB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:37,518 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=C58D17288BCE4D37A73A28582D0C6C8F, vertexId=vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB]
2016-05-17 21:29:37,519 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:37,519 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E4FE813CD0B94F5E9C78531CEE121BFB, vertexId=vertex_1463520577350_0001_1_01, initRequestedTime=1463520577512, initedTime=1463520577519, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:37,520 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:37,520 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:37,520 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {D7B14FB1939947CEB4C1BB1626CD170E=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:37,520 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0] parallelism set to 1
2016-05-17 21:29:37,520 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:37,521 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520577350_0001_1_00, eventCount=1
2016-05-17 21:29:37,521 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:37,521 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FAD9703BC5A9450A9AE06EE9D076BDB0, vertexId=vertex_1463520577350_0001_1_00, initRequestedTime=1463520577510, initedTime=1463520577521, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:37,521 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0] with distanceFromRoot: 0
2016-05-17 21:29:37,522 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:37,522 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FAD9703BC5A9450A9AE06EE9D076BDB0
2016-05-17 21:29:37,522 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0]
2016-05-17 21:29:37,522 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:37,523 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520577350_0001_1_00, startRequestedTime=1463520577522, startedTime=1463520577522
2016-05-17 21:29:37,523 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:37,523 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:37,523 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:TASK_STARTED]: vertexName=FAD9703BC5A9450A9AE06EE9D076BDB0, taskId=task_1463520577350_0001_1_00_000000, scheduledTime=1463520577523, launchTime=1463520577523
2016-05-17 21:29:37,524 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520577350_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:37,524 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520577350_0001_1_00 for vertex: vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB] numStartedSources: 1 numSources: 1
2016-05-17 21:29:37,524 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB] with distanceFromRoot: 1
2016-05-17 21:29:37,524 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0]
2016-05-17 21:29:37,525 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FAD9703BC5A9450A9AE06EE9D076BDB0 in vertex: E4FE813CD0B94F5E9C78531CEE121BFB
2016-05-17 21:29:37,525 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E4FE813CD0B94F5E9C78531CEE121BFB with 1 source tasks and 1 pending tasks
2016-05-17 21:29:37,525 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:37,525 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520577350_0001_1_01, startRequestedTime=1463520577524, startedTime=1463520577524
2016-05-17 21:29:37,525 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:37,526 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5CF2D6598A0D447AAC2ED7C0CE155DDA/F335E54EF5E84F65B2E1073D2648511A] every/(1/1) ...blyplatform/0_'field'_ALL, VertexName: FAD9703BC5A9450A9AE06EE9D076BDB0, VertexParallelism: 1, TaskAttemptID:attempt_1463520577350_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D7B14FB1939947CEB4C1BB1626CD170E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E4FE813CD0B94F5E9C78531CEE121BFB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:37,526 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520577350_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:37,526 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:37,527 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:37,527 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520577350_0001, with dag id: dag_1463520577350_0001_1
2016-05-17 21:29:37,527 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:37,527 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:37,528 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520577350_0001_00_000001
2016-05-17 21:29:37,528 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577350_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:37,529 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520577350_0001_00_000001
2016-05-17 21:29:37,530 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:37,530 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577350_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:37,530 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:37,531 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520577350_0001_00_000001, launchTime=1463520577530
2016-05-17 21:29:37,532 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520577350_0001_00_000001
2016-05-17 21:29:37,532 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520577350_0001_1_00_000000_0] to container: [container_1463520577350_0001_00_000001]
2016-05-17 21:29:37,532 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577350_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:37,533 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520577350_0001_1_00_000000_0] started. Is using containerId: [container_1463520577350_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:37,533 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FAD9703BC5A9450A9AE06EE9D076BDB0, taskAttemptId=attempt_1463520577350_0001_1_00_000000_0, startTime=1463520577533, containerId=container_1463520577350_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520577350_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:37,533 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520577350_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:37,533 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520577350_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:37,533 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:37,533 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520577350_0001_00_000001 given task: attempt_1463520577350_0001_1_00_000000_0
2016-05-17 21:29:37,534 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520577350_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520577350_0001_1_00_000000_0
2016-05-17 21:29:37 Starting to run new task attempt: attempt_1463520577350_0001_1_00_000000_0
2016-05-17 21:29:37,534 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520577350_0001_1_00_000000_0
2016-05-17 21:29:37,534 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:37,534 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:37,534 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:37,534 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:37,534 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:37,535 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:37,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5CF2D6598A0D447AAC2ED7C0CE155DDA/F335E54EF5E84F65B2E1073D2648511A] every/(1/1) ...blyplatform/0_'field'_ALL, VertexName: FAD9703BC5A9450A9AE06EE9D076BDB0, VertexParallelism: 1, TaskAttemptID:attempt_1463520577350_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D7B14FB1939947CEB4C1BB1626CD170E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E4FE813CD0B94F5E9C78531CEE121BFB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:37,535 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:37,535 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520577350_0001_1_00_000000_0
2016-05-17 21:29:37,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:37,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:37,536 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520577350_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:37,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:37,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D7B14FB1939947CEB4C1BB1626CD170E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:37,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:37,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D7B14FB1939947CEB4C1BB1626CD170E
2016-05-17 21:29:37,537 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: D7B14FB1939947CEB4C1BB1626CD170E
2016-05-17 21:29:37,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E4FE813CD0B94F5E9C78531CEE121BFB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:37,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:37,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E4FE813CD0B94F5E9C78531CEE121BFB
2016-05-17 21:29:37,538 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:37,538 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:37,539 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: D7B14FB1939947CEB4C1BB1626CD170E
2016-05-17 21:29:37,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D7B14FB1939947CEB4C1BB1626CD170E
2016-05-17 21:29:37,540 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:37,540 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E4FE813CD0B94F5E9C78531CEE121BFB
2016-05-17 21:29:37,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E4FE813CD0B94F5E9C78531CEE121BFB
2016-05-17 21:29:37,546 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FAD9703BC5A9450A9AE06EE9D076BDB0, ordinal: 0
2016-05-17 21:29:37,547 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FAD9703BC5A9450A9AE06EE9D076BDB0, mem on start (mb), free: 238, total: 358, max: 672
2016-05-17 21:29:37,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:37,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:37,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:37,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:37,547 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:37,547 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:37,547 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:37,547 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:37,548 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:37,548 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:37,548 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D7B14FB1939947CEB4C1BB1626CD170E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:37,548 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E4FE813CD0B94F5E9C78531CEE121BFB, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:37,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:37,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D7B14FB1939947CEB4C1BB1626CD170E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:37,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:37,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:37,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D7B14FB1939947CEB4C1BB1626CD170E
2016-05-17 21:29:37,549 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:37,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D7B14FB1939947CEB4C1BB1626CD170E
2016-05-17 21:29:37,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:37,549 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520577350_0001_1_00_000000_0
2016-05-17 21:29:37,549 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:37,550 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6a4d7764
2016-05-17 21:29:37,551 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6a4d7764
2016-05-17 21:29:37,551 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:37,551 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:37,552 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:37,553 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: D7B14FB1939947CEB4C1BB1626CD170E
2016-05-17 21:29:37,553 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: E8DFB00ABBEA48B98ACCC3E5243BE946
2016-05-17 21:29:37,553 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] E8DFB00ABBEA48B98ACCC3E5243BE946
2016-05-17 21:29:37,554 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:37,558 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:37,558 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@a96b4e7; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@2c3eaf1f; comparator=cascading.tuple.hadoop.util.TupleComparator@475b32ea; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:37,558 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:37,558 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:37,558 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:37,571 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:37,571 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:37,572 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:37,572 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:37,572 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:37,573 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:37,573 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:37,573 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FAD9703BC5A9450A9AE06EE9D076BDB0, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:37,576 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520577350_0001_1_00_000000_0
2016-05-17 21:29:37,576 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FAD9703BC5A9450A9AE06EE9D076BDB0, mem on close (mb), free: 136, total: 358, max: 672
2016-05-17 21:29:37,576 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:37,576 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:37,576 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:37,576 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:37,579 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:37 Completed running task attempt: attempt_1463520577350_0001_1_00_000000_0
2016-05-17 21:29:37,582 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520577350_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:37,582 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520577350_0001_1_00, eventCount=1
2016-05-17 21:29:37,583 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FAD9703BC5A9450A9AE06EE9D076BDB0, taskAttemptId=attempt_1463520577350_0001_1_00_000000_0, startTime=1463520577533, finishTime=1463520577582, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520577574, Process_Duration=2, Process_End_Time=1463520577576, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:37,583 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520577350_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:37,583 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:37,583 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520577350_0001_1_00_000000_0
2016-05-17 21:29:37,583 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:37,584 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:37,584 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:TASK_FINISHED]: vertexName=FAD9703BC5A9450A9AE06EE9D076BDB0, taskId=task_1463520577350_0001_1_00_000000, startTime=1463520577533, finishTime=1463520577583, timeTaken=50, status=SUCCEEDED, successfulAttemptID=attempt_1463520577350_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520577574, Process_Duration=2, Process_End_Time=1463520577576, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:37,584 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520577350_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:37,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520577574, Process_Duration=2, Process_End_Time=1463520577576, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:37,585 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0] : 1
2016-05-17 21:29:37,585 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:37,585 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:37,585 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0]
2016-05-17 21:29:37,585 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520577350_0001_00_000001
2016-05-17 21:29:37,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:37,586 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:VERTEX_FINISHED]: vertexName=FAD9703BC5A9450A9AE06EE9D076BDB0, vertexId=vertex_1463520577350_0001_1_00, initRequestedTime=1463520577510, initedTime=1463520577521, startRequestedTime=1463520577522, startedTime=1463520577522, finishTime=1463520577585, timeTaken=63, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520577574, Process_Duration=2, Process_End_Time=1463520577576, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520577533, firstTasksToStart=[ task_1463520577350_0001_1_00_000000 ], lastTaskFinishTime=1463520577582, lastTasksToFinish=[ task_1463520577350_0001_1_00_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520577350_0001_1_00_000000 ], longestDurationTasks=[ task_1463520577350_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:37,586 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:37,586 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577350_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:37,586 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520577350_0001_00_000001, stoppedTime=1463520577586, exitStatus=0
2016-05-17 21:29:37,587 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577350_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:37,587 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB] attempt: attempt_1463520577350_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:37,587 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E4FE813CD0B94F5E9C78531CEE121BFB
2016-05-17 21:29:37,587 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB]
2016-05-17 21:29:37,587 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:37,587 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520577350_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:37,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:37,588 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:TASK_STARTED]: vertexName=E4FE813CD0B94F5E9C78531CEE121BFB, taskId=task_1463520577350_0001_1_01_000000, scheduledTime=1463520577587, launchTime=1463520577587
2016-05-17 21:29:37,588 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:37,588 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520577350_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:37,588 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB]
2016-05-17 21:29:37,588 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577350_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:37,588 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5CF2D6598A0D447AAC2ED7C0CE155DDA/F335E54EF5E84F65B2E1073D2648511A] every/(1/1) ...blyplatform/0_'field'_ALL, VertexName: E4FE813CD0B94F5E9C78531CEE121BFB, VertexParallelism: 1, TaskAttemptID:attempt_1463520577350_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FAD9703BC5A9450A9AE06EE9D076BDB0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C58D17288BCE4D37A73A28582D0C6C8F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:37,588 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520577350_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:37,589 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:37,589 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520577350_0001_00_000002
2016-05-17 21:29:37,589 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577350_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:37,591 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520577350_0001_00_000002
2016-05-17 21:29:37,591 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:37,591 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577350_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:37,592 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:37,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520577350_0001_00_000002, launchTime=1463520577592
2016-05-17 21:29:37,592 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520577350_0001_00_000002
2016-05-17 21:29:37,593 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520577350_0001_1_01_000000_0] to container: [container_1463520577350_0001_00_000002]
2016-05-17 21:29:37,593 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577350_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:37,593 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520577350_0001_1_01_000000_0] started. Is using containerId: [container_1463520577350_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:37,593 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E4FE813CD0B94F5E9C78531CEE121BFB, taskAttemptId=attempt_1463520577350_0001_1_01_000000_0, startTime=1463520577593, containerId=container_1463520577350_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520577350_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:37,593 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:37,594 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520577350_0001_00_000002 given task: attempt_1463520577350_0001_1_01_000000_0
2016-05-17 21:29:37,593 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520577350_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:37,594 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520577350_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:37,594 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520577350_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520577350_0001_1_01_000000_0
2016-05-17 21:29:37 Starting to run new task attempt: attempt_1463520577350_0001_1_01_000000_0
2016-05-17 21:29:37,595 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520577350_0001_1_01_000000_0
2016-05-17 21:29:37,595 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:37,595 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:37,595 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:37,595 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:37,595 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:37,596 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:37,596 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520577350_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:37,596 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520577350_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:37,596 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520577350_0001_00_000001, Exiting
2016-05-17 21:29:37,596 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520577350_0001_00_000001
2016-05-17 21:29:37,596 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520577350_0001_00_000001
2016-05-17 21:29:37,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5CF2D6598A0D447AAC2ED7C0CE155DDA/F335E54EF5E84F65B2E1073D2648511A] every/(1/1) ...blyplatform/0_'field'_ALL, VertexName: E4FE813CD0B94F5E9C78531CEE121BFB, VertexParallelism: 1, TaskAttemptID:attempt_1463520577350_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FAD9703BC5A9450A9AE06EE9D076BDB0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C58D17288BCE4D37A73A28582D0C6C8F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:37,597 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:37,597 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520577350_0001_1_01_000000_0
2016-05-17 21:29:37,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:37,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:37,598 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520577350_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:37,598 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520577350_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:37,599 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520577350_0001_00_000001 completed successfully
2016-05-17 21:29:37,599 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577350_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:37,598 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FAD9703BC5A9450A9AE06EE9D076BDB0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:37,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:37,600 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FAD9703BC5A9450A9AE06EE9D076BDB0
2016-05-17 21:29:37,601 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:37,601 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:37,601 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FAD9703BC5A9450A9AE06EE9D076BDB0
2016-05-17 21:29:37,601 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FAD9703BC5A9450A9AE06EE9D076BDB0
2016-05-17 21:29:37,601 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:37,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C58D17288BCE4D37A73A28582D0C6C8F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:37,604 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:37,610 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E4FE813CD0B94F5E9C78531CEE121BFB, ordinal: 1
2016-05-17 21:29:37,611 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4FE813CD0B94F5E9C78531CEE121BFB, mem on start (mb), free: 127, total: 358, max: 672
2016-05-17 21:29:37,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:37,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:37,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:37,604 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:37,611 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C58D17288BCE4D37A73A28582D0C6C8F
2016-05-17 21:29:37,611 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:37,611 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: C58D17288BCE4D37A73A28582D0C6C8F
2016-05-17 21:29:37,624 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:37,624 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C58D17288BCE4D37A73A28582D0C6C8F
2016-05-17 21:29:37,624 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:37,624 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:37,624 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:37,624 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:37,625 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:37,625 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:37,625 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:37,625 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FAD9703BC5A9450A9AE06EE9D076BDB0, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:37,625 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C58D17288BCE4D37A73A28582D0C6C8F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:37,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:37,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FAD9703BC5A9450A9AE06EE9D076BDB0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:37,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:37,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:37,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FAD9703BC5A9450A9AE06EE9D076BDB0
2016-05-17 21:29:37,626 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:37,627 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:37,627 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FAD9703BC5A9450A9AE06EE9D076BDB0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:37,627 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:37,627 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:37,627 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:37,628 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FAD9703BC5A9450A9AE06EE9D076BDB0
2016-05-17 21:29:37,629 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:37,629 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520577350_0001_1_01_000000_0
2016-05-17 21:29:37,636 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:37,636 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: E8DFB00ABBEA48B98ACCC3E5243BE946
2016-05-17 21:29:37,636 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_ALL"], id: C58D17288BCE4D37A73A28582D0C6C8F
2016-05-17 21:29:37,637 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_ALL"]
2016-05-17 21:29:37,637 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] E8DFB00ABBEA48B98ACCC3E5243BE946, for 1 inputs
2016-05-17 21:29:37,637 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520577350_0001_1_00_000000_0_10135, runDuration: 42674, ]
2016-05-17 21:29:37,638 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520577350_0001_1_00_000000_0_10135] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520577638, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:37,639 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FAD9703BC5A9450A9AE06EE9D076BDB0
2016-05-17 21:29:37,639 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:37,639 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [FAD9703BC5A9450A9AE06EE9D076BDB0] #23 in 1ms
2016-05-17 21:29:37,639 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FAD9703BC5A9450A9AE06EE9D076BDB0] #23, status:false, isInterrupted:false
2016-05-17 21:29:37,640 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:37,641 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:37,642 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:37,642 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:37,642 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:37,642 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:37,643 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E4FE813CD0B94F5E9C78531CEE121BFB, all 1 inputs ready in: 00:00:00.006
2016-05-17 21:29:37,643 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FAD9703BC5A9450A9AE06EE9D076BDB0
2016-05-17 21:29:37,644 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:37,646 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@79718472; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@bae96a8
2016-05-17 21:29:37,649 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:37,651 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:37,651 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:37,652 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520577350_0001_1_01_000000_0
2016-05-17 21:29:37,652 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520577350_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:37,653 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520577350_0001_1_01_000000_0
2016-05-17 21:29:37,653 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4FE813CD0B94F5E9C78531CEE121BFB, mem on close (mb), free: 122, total: 358, max: 672
2016-05-17 21:29:37,653 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FAD9703BC5A9450A9AE06EE9D076BDB0
2016-05-17 21:29:37,654 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:37,655 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:37,655 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:37,655 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:37,655 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:37,656 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:37 Completed running task attempt: attempt_1463520577350_0001_1_01_000000_0
2016-05-17 21:29:37,657 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520577350_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:37,658 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:37,658 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E4FE813CD0B94F5E9C78531CEE121BFB, taskAttemptId=attempt_1463520577350_0001_1_01_000000_0, startTime=1463520577593, finishTime=1463520577657, timeTaken=64, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520577643, Process_Duration=10, Process_End_Time=1463520577653, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:37,658 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520577350_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:37,658 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520577350_0001_1_01_000000_0
2016-05-17 21:29:37,658 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:37,659 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:37,659 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:TASK_FINISHED]: vertexName=E4FE813CD0B94F5E9C78531CEE121BFB, taskId=task_1463520577350_0001_1_01_000000, startTime=1463520577593, finishTime=1463520577658, timeTaken=65, status=SUCCEEDED, successfulAttemptID=attempt_1463520577350_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520577643, Process_Duration=10, Process_End_Time=1463520577653, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:37,659 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520577350_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:37,659 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB] : 1
2016-05-17 21:29:37,659 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB]
2016-05-17 21:29:37,660 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:VERTEX_FINISHED]: vertexName=E4FE813CD0B94F5E9C78531CEE121BFB, vertexId=vertex_1463520577350_0001_1_01, initRequestedTime=1463520577512, initedTime=1463520577519, startRequestedTime=1463520577524, startedTime=1463520577524, finishTime=1463520577659, timeTaken=135, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520577643, Process_Duration=10, Process_End_Time=1463520577653, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520577593, firstTasksToStart=[ task_1463520577350_0001_1_01_000000 ], lastTaskFinishTime=1463520577657, lastTasksToFinish=[ task_1463520577350_0001_1_01_000000 ], minTaskDuration=64, maxTaskDuration=64, avgTaskDuration=64.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520577350_0001_1_01_000000 ], longestDurationTasks=[ task_1463520577350_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:37,660 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:37,661 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577350_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:37,661 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520577350_0001_00_000002, stoppedTime=1463520577661, exitStatus=0
2016-05-17 21:29:37,661 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577350_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:37,661 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:37,660 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:37,662 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520577350_0001_1
2016-05-17 21:29:37,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520577350_0001_1
2016-05-17 21:29:37,662 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520577350_0001_1_00 [FAD9703BC5A9450A9AE06EE9D076BDB0]
2016-05-17 21:29:37,663 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: C58D17288BCE4D37A73A28582D0C6C8F for vertex: vertex_1463520577350_0001_1_01 [E4FE813CD0B94F5E9C78531CEE121BFB]
2016-05-17 21:29:37,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:37,666 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=375914496, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520577643, Process_Duration=10, Process_End_Time=1463520577653, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:37,667 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:37,667 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:37,668 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520577350_0001_00_000002
2016-05-17 21:29:37,668 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520577350_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:37,668 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520577350_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:37,669 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520577350_0001_00_000002, Exiting
2016-05-17 21:29:37,669 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520577350_0001_00_000002
2016-05-17 21:29:37,669 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520577350_0001_00_000002
2016-05-17 21:29:37,670 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520577350_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:37,670 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520577350_0001_00_000002 completed successfully
2016-05-17 21:29:37,673 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520577350_0001_1, queueSize=0
2016-05-17 21:29:37,673 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:37,673 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577350_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520577350_0001_1, startTime=1463520577509, finishTime=1463520577671, timeTaken=162, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=451, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=751828992, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041155217, Process_Duration=12, Process_End_Time=2927041155229, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:37,673 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520577350_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:37,673 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520577350_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:37,674 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577350_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:37,674 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520577350_0001_00_000002
2016-05-17 21:29:37 Completed Dag: dag_1463520577350_0001_1
2016-05-17 21:29:37,674 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520577350_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:37,674 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520577350_0001_1_post
2016-05-17 21:29:37,674 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:37,674 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:37,681 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/0_'field'_ALL, applicationId=application_1463520577350_0001
2016-05-17 21:29:37,683 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:37,683 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:37,683 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:37,683 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:37,683 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:37,684 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:37,685 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:37,685 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:37,685 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:37,685 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:37,685 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:37,685 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:37,686 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:37,691 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5CF2D6598A0D447AAC2ED7C0CE155DDA/F335E54EF5E84F65B2E1073D2648511A/.tez/application_1463520577350_0001
2016-05-17 21:29:37,692 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_ALL/_temporary
2016-05-17 21:29:37,692 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.360
2016-05-17 21:29:37,770 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_ALL
2016-05-17 21:29:37,786 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:37,786 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:37,790 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:37,790 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:37,815 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 21:29:37,817 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-17 21:29:37,818 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:37,818 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:37,818 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_REPLACE
2016-05-17 21:29:37,837 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:37,837 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:37,870 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 21:29:37,875 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-17 21:29:37,876 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:37,876 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:37,876 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:37,889 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:37,895 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_SWAP
2016-05-17 21:29:37,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:37,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:37,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_SWAP"]
2016-05-17 21:29:37,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:37,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:37,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:37,940 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...lyplatform/0_'field'_SWAP
2016-05-17 21:29:37,940 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:37,951 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:37,955 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:37,955 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:37,955 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:37,964 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/33AB85BD18FC4F1284092F789ABD923B/BDC02CB5214046E6B64AD4B42E5C2CBB/.tez/application_1463520577955_0001 doesn't exist and is created
2016-05-17 21:29:37,979 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:37,979 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:37,979 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/33AB85BD18FC4F1284092F789ABD923B/BDC02CB5214046E6B64AD4B42E5C2CBB/.tez/application_1463520577955_0001
2016-05-17 21:29:37,982 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520577955_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:37,990 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:37,991 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:37,992 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:37,992 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:37,992 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/33AB85BD18FC4F1284092F789ABD923B/BDC02CB5214046E6B64AD4B42E5C2CBB/.tez/application_1463520577955_0001/localmode-log-dir
2016-05-17 21:29:37,993 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/33AB85BD18FC4F1284092F789ABD923B/BDC02CB5214046E6B64AD4B42E5C2CBB/.tez/application_1463520577955_0001/localmode-log-dir/history.txt.appattempt_1463520577955_0001_000000, maxErrors=10
2016-05-17 21:29:37,993 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:37,993 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520577955_0001_000000, appSubmitTime=1463520577981, launchTime=1463520577981
2016-05-17 21:29:37,995 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:37,997 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:38,001 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:45444
2016-05-17 21:29:38,001 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:38,004 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:38,004 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:38,006 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520577955_0001_000000, startTime=1463520578006
2016-05-17 21:29:38,006 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:38,006 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:38,079 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:38,079 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:38,079 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_'field'_SWAP, applicationId=application_1463520577955_0001, dagName=[33AB85BD18FC4F1284092F789ABD923B/BDC02CB5214046E6B64AD4B42E5C2CBB] every/(1/1) ...lyplatform/0_'field'_SWAP
2016-05-17 21:29:38,094 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [33AB85BD18FC4F1284092F789ABD923B/BDC02CB5214046E6B64AD4B42E5C2CBB] every/(1/1) ...lyplatform/0_'field'_SWAP
2016-05-17 21:29:38,095 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520577955_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/33AB85BD18FC4F1284092F789ABD923B/BDC02CB5214046E6B64AD4B42E5C2CBB/.tez/application_1463520577955_0001/localmode-log-dir/dag_1463520577955_0001_1.dot
2016-05-17 21:29:38,095 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/33AB85BD18FC4F1284092F789ABD923B/BDC02CB5214046E6B64AD4B42E5C2CBB/.tez/application_1463520577955_0001/localmode-log-dir/dag_1463520577955_0001_1-tez-dag.pb.txt
2016-05-17 21:29:38,102 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520577955_0001_1
2016-05-17 21:29:38,102 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:38,102 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [33AB85BD18FC4F1284092F789ABD923B/BDC02CB5214046E6B64AD4B42E5C2CBB] every/(1/1) ...lyplatform/0_'field'_SWAP
2016-05-17 21:29:38 Running Dag: dag_1463520577955_0001_1
2016-05-17 21:29:38,114 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520577955_0001_1, submitTime=1463520578094
2016-05-17 21:29:38,114 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:38,115 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D0938F8B0B8E49FB843D0B7D5E511FE4
2016-05-17 21:29:38,116 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:38,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520577955_0001_1, initTime=1463520578115
2016-05-17 21:29:38,117 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520577955_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:38,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:DAG_STARTED]: dagID=dag_1463520577955_0001_1, startTime=1463520578117
2016-05-17 21:29:38,117 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:38,117 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:38,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:38,117 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520577955_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:38,118 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BD595972540B4A1AA0DED59EA9920C43 : {961110FA978A4209ADBD563B0AE5620C={InputName=961110FA978A4209ADBD563B0AE5620C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:38,118 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 961110FA978A4209ADBD563B0AE5620C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:38,118 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43]
2016-05-17 21:29:38,118 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43]
2016-05-17 21:29:38,118 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43]
2016-05-17 21:29:38,119 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_'field'_SWAP, applicationId=application_1463520577955_0001, dagName=[33AB85BD18FC4F1284092F789ABD923B/BDC02CB5214046E6B64AD4B42E5C2CBB] every/(1/1) ...lyplatform/0_'field'_SWAP
2016-05-17 21:29:38,119 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43]. Starting root input initializers: 1
2016-05-17 21:29:38,119 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:38,119 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4]
2016-05-17 21:29:38,121 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 961110FA978A4209ADBD563B0AE5620C on vertex vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43]
2016-05-17 21:29:38,122 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:38,122 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4]
2016-05-17 21:29:38,122 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4]
2016-05-17 21:29:38,122 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4]
2016-05-17 21:29:38,123 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1C2A9602BF5F442D9B87AE7AC4B408A2, vertexId=vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:38,123 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1C2A9602BF5F442D9B87AE7AC4B408A2, vertexId=vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4]
2016-05-17 21:29:38,123 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:38,124 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 961110FA978A4209ADBD563B0AE5620C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:38,124 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:38,124 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:38,125 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1C2A9602BF5F442D9B87AE7AC4B408A2, vertexName=D0938F8B0B8E49FB843D0B7D5E511FE4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:38,125 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1C2A9602BF5F442D9B87AE7AC4B408A2, vertexId=vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4]
2016-05-17 21:29:38,126 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:38,126 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 961110FA978A4209ADBD563B0AE5620C on vertex vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43]
2016-05-17 21:29:38,126 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:38,127 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D0938F8B0B8E49FB843D0B7D5E511FE4, vertexId=vertex_1463520577955_0001_1_01, initRequestedTime=1463520578119, initedTime=1463520578126, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:38,127 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:38,127 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:38,127 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {961110FA978A4209ADBD563B0AE5620C=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:38,127 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43] parallelism set to 1
2016-05-17 21:29:38,127 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:38,128 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520577955_0001_1_00, eventCount=1
2016-05-17 21:29:38,128 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:38,128 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BD595972540B4A1AA0DED59EA9920C43, vertexId=vertex_1463520577955_0001_1_00, initRequestedTime=1463520578118, initedTime=1463520578128, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:38,129 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43] with distanceFromRoot: 0
2016-05-17 21:29:38,129 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:38,129 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BD595972540B4A1AA0DED59EA9920C43
2016-05-17 21:29:38,129 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43]
2016-05-17 21:29:38,129 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:38,129 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520577955_0001_1_00, startRequestedTime=1463520578129, startedTime=1463520578129
2016-05-17 21:29:38,130 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:38,130 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:38,130 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:TASK_STARTED]: vertexName=BD595972540B4A1AA0DED59EA9920C43, taskId=task_1463520577955_0001_1_00_000000, scheduledTime=1463520578130, launchTime=1463520578130
2016-05-17 21:29:38,130 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520577955_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:38,130 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520577955_0001_1_00 for vertex: vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4] numStartedSources: 1 numSources: 1
2016-05-17 21:29:38,130 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4] with distanceFromRoot: 1
2016-05-17 21:29:38,131 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43]
2016-05-17 21:29:38,131 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BD595972540B4A1AA0DED59EA9920C43 in vertex: D0938F8B0B8E49FB843D0B7D5E511FE4
2016-05-17 21:29:38,131 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D0938F8B0B8E49FB843D0B7D5E511FE4 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:38,131 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:38,131 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520577955_0001_1_01, startRequestedTime=1463520578131, startedTime=1463520578131
2016-05-17 21:29:38,132 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:38,132 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [33AB85BD18FC4F1284092F789ABD923B/BDC02CB5214046E6B64AD4B42E5C2CBB] every/(1/1) ...lyplatform/0_'field'_SWAP, VertexName: BD595972540B4A1AA0DED59EA9920C43, VertexParallelism: 1, TaskAttemptID:attempt_1463520577955_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=961110FA978A4209ADBD563B0AE5620C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D0938F8B0B8E49FB843D0B7D5E511FE4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:38,132 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520577955_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:38,132 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:38,133 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:38,133 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:38,133 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:38,133 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520577955_0001_00_000001
2016-05-17 21:29:38,134 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577955_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:38,135 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520577955_0001_00_000001
2016-05-17 21:29:38,135 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:38,136 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577955_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:38,136 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:38,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520577955_0001_00_000001, launchTime=1463520578136
2016-05-17 21:29:38,137 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520577955_0001_00_000001
2016-05-17 21:29:38,137 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520577955_0001_1_00_000000_0] to container: [container_1463520577955_0001_00_000001]
2016-05-17 21:29:38,137 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577955_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:38,138 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520577955_0001_1_00_000000_0] started. Is using containerId: [container_1463520577955_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:38,138 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BD595972540B4A1AA0DED59EA9920C43, taskAttemptId=attempt_1463520577955_0001_1_00_000000_0, startTime=1463520578137, containerId=container_1463520577955_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520577955_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:38,138 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520577955_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:38,138 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520577955_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:38,138 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520577955_0001, with dag id: dag_1463520577955_0001_1
2016-05-17 21:29:38,138 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520577955_0001_00_000001 given task: attempt_1463520577955_0001_1_00_000000_0
2016-05-17 21:29:38,138 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:38,139 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520577955_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520577955_0001_1_00_000000_0
2016-05-17 21:29:38 Starting to run new task attempt: attempt_1463520577955_0001_1_00_000000_0
2016-05-17 21:29:38,139 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520577955_0001_1_00_000000_0
2016-05-17 21:29:38,139 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:38,139 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:38,139 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:38,140 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:38,140 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:38,140 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:38,140 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [33AB85BD18FC4F1284092F789ABD923B/BDC02CB5214046E6B64AD4B42E5C2CBB] every/(1/1) ...lyplatform/0_'field'_SWAP, VertexName: BD595972540B4A1AA0DED59EA9920C43, VertexParallelism: 1, TaskAttemptID:attempt_1463520577955_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=961110FA978A4209ADBD563B0AE5620C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D0938F8B0B8E49FB843D0B7D5E511FE4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:38,140 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:38,140 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520577955_0001_1_00_000000_0
2016-05-17 21:29:38,141 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:38,141 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:38,141 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520577955_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:38,142 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=961110FA978A4209ADBD563B0AE5620C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:38,142 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:38,143 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 961110FA978A4209ADBD563B0AE5620C
2016-05-17 21:29:38,143 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 961110FA978A4209ADBD563B0AE5620C
2016-05-17 21:29:38,145 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:38,145 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 961110FA978A4209ADBD563B0AE5620C
2016-05-17 21:29:38,145 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 961110FA978A4209ADBD563B0AE5620C
2016-05-17 21:29:38,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:38,148 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:38,149 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D0938F8B0B8E49FB843D0B7D5E511FE4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:38,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:38,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D0938F8B0B8E49FB843D0B7D5E511FE4
2016-05-17 21:29:38,151 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:38,152 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D0938F8B0B8E49FB843D0B7D5E511FE4
2016-05-17 21:29:38,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D0938F8B0B8E49FB843D0B7D5E511FE4
2016-05-17 21:29:38,154 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BD595972540B4A1AA0DED59EA9920C43, ordinal: 0
2016-05-17 21:29:38,154 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BD595972540B4A1AA0DED59EA9920C43, mem on start (mb), free: 219, total: 360, max: 672
2016-05-17 21:29:38,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:38,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:38,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:38,155 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:38,155 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:38,155 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:38,156 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:38,156 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:38,156 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:38,156 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:38,156 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 961110FA978A4209ADBD563B0AE5620C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:38,156 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D0938F8B0B8E49FB843D0B7D5E511FE4, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:38,156 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:38,157 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 961110FA978A4209ADBD563B0AE5620C
2016-05-17 21:29:38,157 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:38,157 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 961110FA978A4209ADBD563B0AE5620C
2016-05-17 21:29:38,157 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 961110FA978A4209ADBD563B0AE5620C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:38,158 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:38,158 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:38,158 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:38,158 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520577955_0001_1_00_000000_0
2016-05-17 21:29:38,159 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:38,159 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6d575fa5
2016-05-17 21:29:38,160 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6d575fa5
2016-05-17 21:29:38,160 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:38,160 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:38,161 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:38,162 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 961110FA978A4209ADBD563B0AE5620C
2016-05-17 21:29:38,162 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: E259FAE18C6241DCBD9DB6E1203AE918
2016-05-17 21:29:38,162 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] E259FAE18C6241DCBD9DB6E1203AE918
2016-05-17 21:29:38,163 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:38,167 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:38,167 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@43c781f3; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@5f249b48; comparator=cascading.tuple.hadoop.util.TupleComparator@37584f34; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:38,167 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:38,167 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:38,167 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:38,180 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:38,181 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:38,181 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:38,181 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:38,181 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:38,182 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:38,182 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:38,182 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BD595972540B4A1AA0DED59EA9920C43, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:38,184 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520577955_0001_1_00_000000_0
2016-05-17 21:29:38,185 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BD595972540B4A1AA0DED59EA9920C43, mem on close (mb), free: 117, total: 360, max: 672
2016-05-17 21:29:38,185 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:38,185 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:38,185 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:38,185 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:38,188 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:38 Completed running task attempt: attempt_1463520577955_0001_1_00_000000_0
2016-05-17 21:29:38,190 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520577955_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:38,191 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520577955_0001_1_00, eventCount=1
2016-05-17 21:29:38,191 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BD595972540B4A1AA0DED59EA9920C43, taskAttemptId=attempt_1463520577955_0001_1_00_000000_0, startTime=1463520578137, finishTime=1463520578191, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520578182, Process_Duration=2, Process_End_Time=1463520578184, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:38,191 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520577955_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:38,191 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520577955_0001_1_00_000000_0
2016-05-17 21:29:38,192 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:38,192 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:TASK_FINISHED]: vertexName=BD595972540B4A1AA0DED59EA9920C43, taskId=task_1463520577955_0001_1_00_000000, startTime=1463520578137, finishTime=1463520578192, timeTaken=55, status=SUCCEEDED, successfulAttemptID=attempt_1463520577955_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520578182, Process_Duration=2, Process_End_Time=1463520578184, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:38,192 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:38,192 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520577955_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:38,193 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43] : 1
2016-05-17 21:29:38,193 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43]
2016-05-17 21:29:38,193 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:38,193 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:38,193 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:VERTEX_FINISHED]: vertexName=BD595972540B4A1AA0DED59EA9920C43, vertexId=vertex_1463520577955_0001_1_00, initRequestedTime=1463520578118, initedTime=1463520578128, startRequestedTime=1463520578129, startedTime=1463520578129, finishTime=1463520578193, timeTaken=64, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520578182, Process_Duration=2, Process_End_Time=1463520578184, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520578137, firstTasksToStart=[ task_1463520577955_0001_1_00_000000 ], lastTaskFinishTime=1463520578191, lastTasksToFinish=[ task_1463520577955_0001_1_00_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520577955_0001_1_00_000000 ], longestDurationTasks=[ task_1463520577955_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:38,194 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:38,194 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577955_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:38,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520578182, Process_Duration=2, Process_End_Time=1463520578184, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:38,194 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:38,194 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:38,195 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:38,195 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520577955_0001_00_000001
2016-05-17 21:29:38,195 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520577955_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:38,195 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520577955_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:38,195 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520577955_0001_00_000001, Exiting
2016-05-17 21:29:38,195 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520577955_0001_00_000001
2016-05-17 21:29:38,195 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520577955_0001_00_000001
2016-05-17 21:29:38,194 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520577955_0001_00_000001, stoppedTime=1463520578194, exitStatus=0
2016-05-17 21:29:38,195 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577955_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:38,196 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4] attempt: attempt_1463520577955_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:38,196 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D0938F8B0B8E49FB843D0B7D5E511FE4
2016-05-17 21:29:38,196 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4]
2016-05-17 21:29:38,197 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:38,197 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520577955_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:38,197 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:38,198 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:TASK_STARTED]: vertexName=D0938F8B0B8E49FB843D0B7D5E511FE4, taskId=task_1463520577955_0001_1_01_000000, scheduledTime=1463520578197, launchTime=1463520578197
2016-05-17 21:29:38,198 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520577955_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:38,198 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4]
2016-05-17 21:29:38,198 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577955_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:38,198 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [33AB85BD18FC4F1284092F789ABD923B/BDC02CB5214046E6B64AD4B42E5C2CBB] every/(1/1) ...lyplatform/0_'field'_SWAP, VertexName: D0938F8B0B8E49FB843D0B7D5E511FE4, VertexParallelism: 1, TaskAttemptID:attempt_1463520577955_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BD595972540B4A1AA0DED59EA9920C43, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1C2A9602BF5F442D9B87AE7AC4B408A2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:38,198 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520577955_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:38,199 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:38,199 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520577955_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:38,200 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520577955_0001_00_000001 completed successfully
2016-05-17 21:29:38,199 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520577955_0001_00_000002
2016-05-17 21:29:38,200 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577955_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:38,200 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577955_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:38,201 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520577955_0001_00_000002
2016-05-17 21:29:38,201 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:38,203 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577955_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:38,203 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:38,204 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520577955_0001_00_000002, launchTime=1463520578203
2016-05-17 21:29:38,204 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520577955_0001_00_000002
2016-05-17 21:29:38,205 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520577955_0001_1_01_000000_0] to container: [container_1463520577955_0001_00_000002]
2016-05-17 21:29:38,205 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577955_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:38,205 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520577955_0001_1_01_000000_0] started. Is using containerId: [container_1463520577955_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:38,205 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D0938F8B0B8E49FB843D0B7D5E511FE4, taskAttemptId=attempt_1463520577955_0001_1_01_000000_0, startTime=1463520578205, containerId=container_1463520577955_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520577955_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:38,205 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520577955_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:38,206 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520577955_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:38,206 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:38,206 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520577955_0001_00_000002 given task: attempt_1463520577955_0001_1_01_000000_0
2016-05-17 21:29:38,206 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520577955_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520577955_0001_1_01_000000_0
2016-05-17 21:29:38 Starting to run new task attempt: attempt_1463520577955_0001_1_01_000000_0
2016-05-17 21:29:38,206 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520577955_0001_1_01_000000_0
2016-05-17 21:29:38,209 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:38,209 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:38,209 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:38,209 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:38,209 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:38,210 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:38,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [33AB85BD18FC4F1284092F789ABD923B/BDC02CB5214046E6B64AD4B42E5C2CBB] every/(1/1) ...lyplatform/0_'field'_SWAP, VertexName: D0938F8B0B8E49FB843D0B7D5E511FE4, VertexParallelism: 1, TaskAttemptID:attempt_1463520577955_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BD595972540B4A1AA0DED59EA9920C43, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1C2A9602BF5F442D9B87AE7AC4B408A2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:38,210 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:38,211 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520577955_0001_1_01_000000_0
2016-05-17 21:29:38,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:38,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:38,211 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520577955_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:38,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:38,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BD595972540B4A1AA0DED59EA9920C43, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:38,213 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:38,213 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BD595972540B4A1AA0DED59EA9920C43
2016-05-17 21:29:38,214 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:38,215 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:38,215 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BD595972540B4A1AA0DED59EA9920C43
2016-05-17 21:29:38,215 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:38,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BD595972540B4A1AA0DED59EA9920C43
2016-05-17 21:29:38,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1C2A9602BF5F442D9B87AE7AC4B408A2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:38,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:38,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1C2A9602BF5F442D9B87AE7AC4B408A2
2016-05-17 21:29:38,218 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:38,218 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1C2A9602BF5F442D9B87AE7AC4B408A2
2016-05-17 21:29:38,224 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D0938F8B0B8E49FB843D0B7D5E511FE4, ordinal: 1
2016-05-17 21:29:38,224 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D0938F8B0B8E49FB843D0B7D5E511FE4, mem on start (mb), free: 110, total: 360, max: 672
2016-05-17 21:29:38,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:38,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:38,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:38,233 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:38,233 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1C2A9602BF5F442D9B87AE7AC4B408A2
2016-05-17 21:29:38,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:38,235 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:38,235 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:38,235 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:38,235 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:38,235 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:38,236 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:38,236 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BD595972540B4A1AA0DED59EA9920C43, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:38,236 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1C2A9602BF5F442D9B87AE7AC4B408A2, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:38,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:38,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BD595972540B4A1AA0DED59EA9920C43
2016-05-17 21:29:38,237 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:38,237 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:38,238 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BD595972540B4A1AA0DED59EA9920C43 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:38,238 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:38,238 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:38,238 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:38,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BD595972540B4A1AA0DED59EA9920C43
2016-05-17 21:29:38,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BD595972540B4A1AA0DED59EA9920C43 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:38,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:38,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:38,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:38,239 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520577955_0001_1_01_000000_0
2016-05-17 21:29:38,263 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520577955_0001_1_00_000000_0_10141, runDuration: 39471, ]
2016-05-17 21:29:38,264 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520577955_0001_1_00_000000_0_10141] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520578264, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:38,265 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:38,265 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: E259FAE18C6241DCBD9DB6E1203AE918
2016-05-17 21:29:38,265 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_SWAP"], id: 1C2A9602BF5F442D9B87AE7AC4B408A2
2016-05-17 21:29:38,266 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_SWAP"]
2016-05-17 21:29:38,266 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] E259FAE18C6241DCBD9DB6E1203AE918, for 1 inputs
2016-05-17 21:29:38,265 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BD595972540B4A1AA0DED59EA9920C43
2016-05-17 21:29:38,267 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:38,267 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [BD595972540B4A1AA0DED59EA9920C43] #24 in 3ms
2016-05-17 21:29:38,267 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BD595972540B4A1AA0DED59EA9920C43] #24, status:false, isInterrupted:false
2016-05-17 21:29:38,268 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:38,269 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:38,269 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:38,270 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:38,270 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:38,270 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:38,270 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BD595972540B4A1AA0DED59EA9920C43
2016-05-17 21:29:38,270 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:38,271 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D0938F8B0B8E49FB843D0B7D5E511FE4, all 1 inputs ready in: 00:00:00.005
2016-05-17 21:29:38,272 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:38,274 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@5dd9f295; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@fc54456
2016-05-17 21:29:38,282 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:38,282 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:38,282 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520577955_0001_1_01_000000_0
2016-05-17 21:29:38,282 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520577955_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:38,283 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520577955_0001_1_01_000000_0
2016-05-17 21:29:38,283 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D0938F8B0B8E49FB843D0B7D5E511FE4, mem on close (mb), free: 313, total: 360, max: 672
2016-05-17 21:29:38,284 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BD595972540B4A1AA0DED59EA9920C43
2016-05-17 21:29:38,284 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:38,285 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:38,285 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:38,285 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:38,285 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:38,285 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:38 Completed running task attempt: attempt_1463520577955_0001_1_01_000000_0
2016-05-17 21:29:38,286 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520577955_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:38,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520578271, Process_Duration=12, Process_End_Time=1463520578283, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:38,287 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:38,287 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D0938F8B0B8E49FB843D0B7D5E511FE4, taskAttemptId=attempt_1463520577955_0001_1_01_000000_0, startTime=1463520578205, finishTime=1463520578287, timeTaken=82, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520578271, Process_Duration=12, Process_End_Time=1463520578283, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:38,287 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520577955_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:38,288 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520577955_0001_1_01_000000_0
2016-05-17 21:29:38,287 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:38,288 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:38,288 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:TASK_FINISHED]: vertexName=D0938F8B0B8E49FB843D0B7D5E511FE4, taskId=task_1463520577955_0001_1_01_000000, startTime=1463520578205, finishTime=1463520578288, timeTaken=83, status=SUCCEEDED, successfulAttemptID=attempt_1463520577955_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520578271, Process_Duration=12, Process_End_Time=1463520578283, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:38,288 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520577955_0001_00_000002
2016-05-17 21:29:38,288 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520577955_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:38,288 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:38,288 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4] : 1
2016-05-17 21:29:38,289 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4]
2016-05-17 21:29:38,289 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:38,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:VERTEX_FINISHED]: vertexName=D0938F8B0B8E49FB843D0B7D5E511FE4, vertexId=vertex_1463520577955_0001_1_01, initRequestedTime=1463520578119, initedTime=1463520578126, startRequestedTime=1463520578131, startedTime=1463520578131, finishTime=1463520578289, timeTaken=158, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520578271, Process_Duration=12, Process_End_Time=1463520578283, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520578205, firstTasksToStart=[ task_1463520577955_0001_1_01_000000 ], lastTaskFinishTime=1463520578287, lastTasksToFinish=[ task_1463520577955_0001_1_01_000000 ], minTaskDuration=82, maxTaskDuration=82, avgTaskDuration=82.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520577955_0001_1_01_000000 ], longestDurationTasks=[ task_1463520577955_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:38,290 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:38,290 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:38,290 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520577955_0001_1
2016-05-17 21:29:38,290 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520577955_0001_1
2016-05-17 21:29:38,290 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 1C2A9602BF5F442D9B87AE7AC4B408A2 for vertex: vertex_1463520577955_0001_1_01 [D0938F8B0B8E49FB843D0B7D5E511FE4]
2016-05-17 21:29:38,291 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:38,303 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520577955_0001_1_00 [BD595972540B4A1AA0DED59EA9920C43]
2016-05-17 21:29:38,305 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520577955_0001_1, queueSize=0
2016-05-17 21:29:38,305 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:38,306 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520577955_0001_1, startTime=1463520578117, finishTime=1463520578304, timeTaken=187, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=396, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=754974720, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=30, MERGE_PHASE_TIME=33, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041156453, Process_Duration=14, Process_End_Time=2927041156467, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:38,306 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520577955_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:38,306 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520577955_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:38,306 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577955_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:38,306 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520577955_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520577955_0001_00_000002, stoppedTime=1463520578306, exitStatus=0
2016-05-17 21:29:38,306 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:38,306 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577955_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:38 Completed Dag: dag_1463520577955_0001_1
2016-05-17 21:29:38,307 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520577955_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:38,308 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520577955_0001_1_post
2016-05-17 21:29:38,308 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:38,308 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:38,308 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520577955_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:38,308 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520577955_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:38,314 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_'field'_SWAP, applicationId=application_1463520577955_0001
2016-05-17 21:29:38,316 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:38,316 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:38,316 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:38,316 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:38,316 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:38,317 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:38,317 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:38,317 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:38,317 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:38,318 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:38,318 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:38,318 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:38,318 INFO task.TezChild (TezChild.java:run(198)) - Interrupted while waiting for new work for container container_1463520577955_0001_00_000002
2016-05-17 21:29:38,318 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520577955_0001_00_000002
2016-05-17 21:29:38,319 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520577955_0001_00_000002
2016-05-17 21:29:38,323 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:38,325 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/33AB85BD18FC4F1284092F789ABD923B/BDC02CB5214046E6B64AD4B42E5C2CBB/.tez/application_1463520577955_0001
2016-05-17 21:29:38,327 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_SWAP/_temporary
2016-05-17 21:29:38,327 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.388
2016-05-17 21:29:38,344 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:38,344 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:38,396 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_SWAP
2016-05-17 21:29:38,416 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:38,417 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:38,447 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 21:29:38,457 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-17 21:29:38,458 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:38,458 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:38,458 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:38,471 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:38,476 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_-1
2016-05-17 21:29:38,517 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:38,518 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:38,518 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_-1"]
2016-05-17 21:29:38,518 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:38,518 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:38,518 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:38,519 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...mblyplatform/0_UNKNOWN_-1
2016-05-17 21:29:38,519 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:38,530 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:38,535 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:38,535 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:38,536 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:38,544 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/36C11CE86D2C4F2FB1DA6ABEC41327D0/ECC8B53579F045B8A868571EB73C599C/.tez/application_1463520578535_0001 doesn't exist and is created
2016-05-17 21:29:38,560 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:38,560 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:38,561 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/36C11CE86D2C4F2FB1DA6ABEC41327D0/ECC8B53579F045B8A868571EB73C599C/.tez/application_1463520578535_0001
2016-05-17 21:29:38,563 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520578535_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:38,573 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:38,573 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:38,575 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:38,575 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:38,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/36C11CE86D2C4F2FB1DA6ABEC41327D0/ECC8B53579F045B8A868571EB73C599C/.tez/application_1463520578535_0001/localmode-log-dir
2016-05-17 21:29:38,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/36C11CE86D2C4F2FB1DA6ABEC41327D0/ECC8B53579F045B8A868571EB73C599C/.tez/application_1463520578535_0001/localmode-log-dir/history.txt.appattempt_1463520578535_0001_000000, maxErrors=10
2016-05-17 21:29:38,578 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:38,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520578535_0001_000000, appSubmitTime=1463520578563, launchTime=1463520578563
2016-05-17 21:29:38,584 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34026
2016-05-17 21:29:38,585 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:38,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:38,588 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:38,588 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:38,589 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:38,591 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520578535_0001_000000, startTime=1463520578591
2016-05-17 21:29:38,591 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:38,591 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:38,661 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:38,661 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:38,661 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/0_UNKNOWN_-1, applicationId=application_1463520578535_0001, dagName=[36C11CE86D2C4F2FB1DA6ABEC41327D0/ECC8B53579F045B8A868571EB73C599C] every/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-05-17 21:29:38,676 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [36C11CE86D2C4F2FB1DA6ABEC41327D0/ECC8B53579F045B8A868571EB73C599C] every/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-05-17 21:29:38,678 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520578535_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/36C11CE86D2C4F2FB1DA6ABEC41327D0/ECC8B53579F045B8A868571EB73C599C/.tez/application_1463520578535_0001/localmode-log-dir/dag_1463520578535_0001_1.dot
2016-05-17 21:29:38,678 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/36C11CE86D2C4F2FB1DA6ABEC41327D0/ECC8B53579F045B8A868571EB73C599C/.tez/application_1463520578535_0001/localmode-log-dir/dag_1463520578535_0001_1-tez-dag.pb.txt
2016-05-17 21:29:38,687 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520578535_0001_1
2016-05-17 21:29:38,687 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:38,687 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [36C11CE86D2C4F2FB1DA6ABEC41327D0/ECC8B53579F045B8A868571EB73C599C] every/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-05-17 21:29:38 Running Dag: dag_1463520578535_0001_1
2016-05-17 21:29:38,699 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:38,699 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520578535_0001_1, submitTime=1463520578676
2016-05-17 21:29:38,700 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BE25195FDF274D109A7160EB0AD4712D
2016-05-17 21:29:38,701 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:38,701 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:38,701 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520578535_0001_1, initTime=1463520578699
2016-05-17 21:29:38,701 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520578535_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:38,702 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:DAG_STARTED]: dagID=dag_1463520578535_0001_1, startTime=1463520578702
2016-05-17 21:29:38,702 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:38,702 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:38,702 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520578535_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:38,703 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B20E9DB1ACA24B1488523AE5E93400A2 : {823EE28EE0C34FD4AED2A328EC4D10D1={InputName=823EE28EE0C34FD4AED2A328EC4D10D1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:38,703 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 823EE28EE0C34FD4AED2A328EC4D10D1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:38,703 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2]
2016-05-17 21:29:38,703 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/0_UNKNOWN_-1, applicationId=application_1463520578535_0001, dagName=[36C11CE86D2C4F2FB1DA6ABEC41327D0/ECC8B53579F045B8A868571EB73C599C] every/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-05-17 21:29:38,703 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2]
2016-05-17 21:29:38,703 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2]
2016-05-17 21:29:38,703 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2]. Starting root input initializers: 1
2016-05-17 21:29:38,704 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:38,704 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 823EE28EE0C34FD4AED2A328EC4D10D1 on vertex vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2]
2016-05-17 21:29:38,704 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D]
2016-05-17 21:29:38,706 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:38,706 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 823EE28EE0C34FD4AED2A328EC4D10D1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:38,707 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:38,707 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D]
2016-05-17 21:29:38,707 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:38,707 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D]
2016-05-17 21:29:38,707 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:38,707 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D]
2016-05-17 21:29:38,708 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=04769812DEE54F748B8E42A563BF2E88, vertexId=vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:38,708 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=04769812DEE54F748B8E42A563BF2E88, vertexId=vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D]
2016-05-17 21:29:38,710 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=04769812DEE54F748B8E42A563BF2E88, vertexName=BE25195FDF274D109A7160EB0AD4712D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:38,710 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=04769812DEE54F748B8E42A563BF2E88, vertexId=vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D]
2016-05-17 21:29:38,711 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:38,711 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:38,711 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BE25195FDF274D109A7160EB0AD4712D, vertexId=vertex_1463520578535_0001_1_01, initRequestedTime=1463520578704, initedTime=1463520578711, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:38,711 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 823EE28EE0C34FD4AED2A328EC4D10D1 on vertex vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2]
2016-05-17 21:29:38,712 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:38,712 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:38,712 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {823EE28EE0C34FD4AED2A328EC4D10D1=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:38,713 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2] parallelism set to 1
2016-05-17 21:29:38,713 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:38,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520578535_0001_1_00, eventCount=1
2016-05-17 21:29:38,713 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:38,714 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B20E9DB1ACA24B1488523AE5E93400A2, vertexId=vertex_1463520578535_0001_1_00, initRequestedTime=1463520578703, initedTime=1463520578713, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:38,714 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2] with distanceFromRoot: 0
2016-05-17 21:29:38,714 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:38,714 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B20E9DB1ACA24B1488523AE5E93400A2
2016-05-17 21:29:38,715 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2]
2016-05-17 21:29:38,715 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:38,715 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520578535_0001_1_00, startRequestedTime=1463520578714, startedTime=1463520578714
2016-05-17 21:29:38,715 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:38,716 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:38,716 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:TASK_STARTED]: vertexName=B20E9DB1ACA24B1488523AE5E93400A2, taskId=task_1463520578535_0001_1_00_000000, scheduledTime=1463520578716, launchTime=1463520578716
2016-05-17 21:29:38,716 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520578535_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:38,717 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520578535_0001_1_00 for vertex: vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D] numStartedSources: 1 numSources: 1
2016-05-17 21:29:38,717 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D] with distanceFromRoot: 1
2016-05-17 21:29:38,717 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2]
2016-05-17 21:29:38,717 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B20E9DB1ACA24B1488523AE5E93400A2 in vertex: BE25195FDF274D109A7160EB0AD4712D
2016-05-17 21:29:38,718 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BE25195FDF274D109A7160EB0AD4712D with 1 source tasks and 1 pending tasks
2016-05-17 21:29:38,718 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:38,718 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520578535_0001_1_01, startRequestedTime=1463520578717, startedTime=1463520578717
2016-05-17 21:29:38,719 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:38,719 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [36C11CE86D2C4F2FB1DA6ABEC41327D0/ECC8B53579F045B8A868571EB73C599C] every/(1/1) ...mblyplatform/0_UNKNOWN_-1, VertexName: B20E9DB1ACA24B1488523AE5E93400A2, VertexParallelism: 1, TaskAttemptID:attempt_1463520578535_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=823EE28EE0C34FD4AED2A328EC4D10D1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BE25195FDF274D109A7160EB0AD4712D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:38,719 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520578535_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:38,719 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:38,720 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:38,720 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:38,721 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:38,721 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520578535_0001, with dag id: dag_1463520578535_0001_1
2016-05-17 21:29:38,721 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520578535_0001_00_000001
2016-05-17 21:29:38,721 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520578535_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:38,722 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520578535_0001_00_000001
2016-05-17 21:29:38,722 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:38,723 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520578535_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:38,723 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:38,724 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520578535_0001_00_000001, launchTime=1463520578723
2016-05-17 21:29:38,724 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520578535_0001_00_000001
2016-05-17 21:29:38,725 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520578535_0001_1_00_000000_0] to container: [container_1463520578535_0001_00_000001]
2016-05-17 21:29:38,725 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520578535_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:38,725 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520578535_0001_1_00_000000_0] started. Is using containerId: [container_1463520578535_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:38,725 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:38,725 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B20E9DB1ACA24B1488523AE5E93400A2, taskAttemptId=attempt_1463520578535_0001_1_00_000000_0, startTime=1463520578725, containerId=container_1463520578535_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520578535_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:38,726 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520578535_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:38,726 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520578535_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:38,726 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520578535_0001_00_000001 given task: attempt_1463520578535_0001_1_00_000000_0
2016-05-17 21:29:38,726 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520578535_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520578535_0001_1_00_000000_0
2016-05-17 21:29:38 Starting to run new task attempt: attempt_1463520578535_0001_1_00_000000_0
2016-05-17 21:29:38,727 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520578535_0001_1_00_000000_0
2016-05-17 21:29:38,727 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:38,727 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:38,727 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:38,727 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:38,727 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:38,727 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:38,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [36C11CE86D2C4F2FB1DA6ABEC41327D0/ECC8B53579F045B8A868571EB73C599C] every/(1/1) ...mblyplatform/0_UNKNOWN_-1, VertexName: B20E9DB1ACA24B1488523AE5E93400A2, VertexParallelism: 1, TaskAttemptID:attempt_1463520578535_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=823EE28EE0C34FD4AED2A328EC4D10D1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BE25195FDF274D109A7160EB0AD4712D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:38,728 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:38,728 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520578535_0001_1_00_000000_0
2016-05-17 21:29:38,728 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:38,728 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:38,728 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520578535_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:38,729 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:38,729 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=823EE28EE0C34FD4AED2A328EC4D10D1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:38,729 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BE25195FDF274D109A7160EB0AD4712D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:38,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:38,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 823EE28EE0C34FD4AED2A328EC4D10D1
2016-05-17 21:29:38,730 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 823EE28EE0C34FD4AED2A328EC4D10D1
2016-05-17 21:29:38,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:38,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BE25195FDF274D109A7160EB0AD4712D
2016-05-17 21:29:38,732 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:38,732 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 823EE28EE0C34FD4AED2A328EC4D10D1
2016-05-17 21:29:38,732 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 823EE28EE0C34FD4AED2A328EC4D10D1
2016-05-17 21:29:38,732 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:38,733 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:38,733 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BE25195FDF274D109A7160EB0AD4712D
2016-05-17 21:29:38,733 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BE25195FDF274D109A7160EB0AD4712D
2016-05-17 21:29:38,736 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B20E9DB1ACA24B1488523AE5E93400A2, ordinal: 0
2016-05-17 21:29:38,736 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B20E9DB1ACA24B1488523AE5E93400A2, mem on start (mb), free: 218, total: 360, max: 672
2016-05-17 21:29:38,736 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:38,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:38,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:38,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:38,737 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:38,737 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:38,737 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:38,738 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:38,738 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:38,738 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:38,738 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 823EE28EE0C34FD4AED2A328EC4D10D1, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:38,738 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BE25195FDF274D109A7160EB0AD4712D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:38,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:38,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 823EE28EE0C34FD4AED2A328EC4D10D1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:38,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:38,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:38,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 823EE28EE0C34FD4AED2A328EC4D10D1
2016-05-17 21:29:38,740 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:38,740 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 823EE28EE0C34FD4AED2A328EC4D10D1
2016-05-17 21:29:38,740 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:38,740 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520578535_0001_1_00_000000_0
2016-05-17 21:29:38,742 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:38,743 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7dd9d603
2016-05-17 21:29:38,743 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:38,744 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 823EE28EE0C34FD4AED2A328EC4D10D1
2016-05-17 21:29:38,744 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 3FBE13DAAB66455A86A646836DC5020C
2016-05-17 21:29:38,744 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 3FBE13DAAB66455A86A646836DC5020C
2016-05-17 21:29:38,745 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:38,744 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7dd9d603
2016-05-17 21:29:38,745 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:38,745 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:38,747 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:38,747 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@263a6e09; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@460e247a; comparator=cascading.tuple.hadoop.util.TupleComparator@4a7ce984; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:38,747 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:38,747 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:38,747 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:38,760 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:38,760 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:38,760 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:38,760 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:38,760 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:38,761 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:38,761 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:38,762 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B20E9DB1ACA24B1488523AE5E93400A2, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:29:38,764 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520578535_0001_1_00_000000_0
2016-05-17 21:29:38,764 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B20E9DB1ACA24B1488523AE5E93400A2, mem on close (mb), free: 115, total: 360, max: 672
2016-05-17 21:29:38,764 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:38,765 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:38,765 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:38,765 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:38,768 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:38 Completed running task attempt: attempt_1463520578535_0001_1_00_000000_0
2016-05-17 21:29:38,771 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520578535_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:38,772 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520578535_0001_1_00, eventCount=1
2016-05-17 21:29:38,772 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:38,772 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B20E9DB1ACA24B1488523AE5E93400A2, taskAttemptId=attempt_1463520578535_0001_1_00_000000_0, startTime=1463520578725, finishTime=1463520578772, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520578762, Process_Duration=2, Process_End_Time=1463520578764, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:38,772 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520578535_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:38,772 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520578535_0001_1_00_000000_0
2016-05-17 21:29:38,773 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:38,773 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:38,773 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:TASK_FINISHED]: vertexName=B20E9DB1ACA24B1488523AE5E93400A2, taskId=task_1463520578535_0001_1_00_000000, startTime=1463520578725, finishTime=1463520578773, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1463520578535_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520578762, Process_Duration=2, Process_End_Time=1463520578764, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:38,773 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520578535_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:38,773 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2] : 1
2016-05-17 21:29:38,774 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2]
2016-05-17 21:29:38,775 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:38,775 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:VERTEX_FINISHED]: vertexName=B20E9DB1ACA24B1488523AE5E93400A2, vertexId=vertex_1463520578535_0001_1_00, initRequestedTime=1463520578703, initedTime=1463520578713, startRequestedTime=1463520578714, startedTime=1463520578714, finishTime=1463520578774, timeTaken=60, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520578762, Process_Duration=2, Process_End_Time=1463520578764, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520578725, firstTasksToStart=[ task_1463520578535_0001_1_00_000000 ], lastTaskFinishTime=1463520578772, lastTasksToFinish=[ task_1463520578535_0001_1_00_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520578535_0001_1_00_000000 ], longestDurationTasks=[ task_1463520578535_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:38,775 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:38,776 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=377487360, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520578762, Process_Duration=2, Process_End_Time=1463520578764, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:38,776 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520578535_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:38,776 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D] attempt: attempt_1463520578535_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:38,776 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:38,777 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:38,777 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BE25195FDF274D109A7160EB0AD4712D
2016-05-17 21:29:38,777 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D]
2016-05-17 21:29:38,777 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520578535_0001_00_000001, stoppedTime=1463520578777, exitStatus=0
2016-05-17 21:29:38,777 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:38,778 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520578535_0001_00_000001
2016-05-17 21:29:38,778 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520578535_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:38,778 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:38,778 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520578535_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:38,779 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:38,779 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:TASK_STARTED]: vertexName=BE25195FDF274D109A7160EB0AD4712D, taskId=task_1463520578535_0001_1_01_000000, scheduledTime=1463520578779, launchTime=1463520578779
2016-05-17 21:29:38,779 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520578535_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:38,779 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520578535_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:38,780 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D]
2016-05-17 21:29:38,780 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520578535_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:38,780 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [36C11CE86D2C4F2FB1DA6ABEC41327D0/ECC8B53579F045B8A868571EB73C599C] every/(1/1) ...mblyplatform/0_UNKNOWN_-1, VertexName: BE25195FDF274D109A7160EB0AD4712D, VertexParallelism: 1, TaskAttemptID:attempt_1463520578535_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B20E9DB1ACA24B1488523AE5E93400A2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=04769812DEE54F748B8E42A563BF2E88, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:38,780 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520578535_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:38,780 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:38,779 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520578535_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:38,781 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520578535_0001_00_000001, Exiting
2016-05-17 21:29:38,781 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520578535_0001_00_000001
2016-05-17 21:29:38,781 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520578535_0001_00_000001
2016-05-17 21:29:38,782 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520578535_0001_00_000002
2016-05-17 21:29:38,782 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520578535_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:38,782 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520578535_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:38,783 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520578535_0001_00_000001 completed successfully
2016-05-17 21:29:38,783 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520578535_0001_00_000002
2016-05-17 21:29:38,784 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520578535_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:38,784 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:38,786 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520578535_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:38,787 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:38,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520578535_0001_00_000002, launchTime=1463520578786
2016-05-17 21:29:38,788 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520578535_0001_00_000002
2016-05-17 21:29:38,788 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520578535_0001_1_01_000000_0] to container: [container_1463520578535_0001_00_000002]
2016-05-17 21:29:38,788 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520578535_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:38,788 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520578535_0001_1_01_000000_0] started. Is using containerId: [container_1463520578535_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:38,789 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BE25195FDF274D109A7160EB0AD4712D, taskAttemptId=attempt_1463520578535_0001_1_01_000000_0, startTime=1463520578788, containerId=container_1463520578535_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520578535_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:38,789 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:38,789 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520578535_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:38,789 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520578535_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:38,789 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520578535_0001_00_000002 given task: attempt_1463520578535_0001_1_01_000000_0
2016-05-17 21:29:38,789 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520578535_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520578535_0001_1_01_000000_0
2016-05-17 21:29:38 Starting to run new task attempt: attempt_1463520578535_0001_1_01_000000_0
2016-05-17 21:29:38,791 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520578535_0001_1_01_000000_0
2016-05-17 21:29:38,791 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:38,791 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:38,791 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:38,792 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:38,792 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:38,792 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:38,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [36C11CE86D2C4F2FB1DA6ABEC41327D0/ECC8B53579F045B8A868571EB73C599C] every/(1/1) ...mblyplatform/0_UNKNOWN_-1, VertexName: BE25195FDF274D109A7160EB0AD4712D, VertexParallelism: 1, TaskAttemptID:attempt_1463520578535_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B20E9DB1ACA24B1488523AE5E93400A2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=04769812DEE54F748B8E42A563BF2E88, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:38,792 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:38,793 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520578535_0001_1_01_000000_0
2016-05-17 21:29:38,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:38,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:38,793 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520578535_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:38,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B20E9DB1ACA24B1488523AE5E93400A2, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:38,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=04769812DEE54F748B8E42A563BF2E88, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:38,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:38,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:38,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B20E9DB1ACA24B1488523AE5E93400A2
2016-05-17 21:29:38,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:38,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 04769812DEE54F748B8E42A563BF2E88
2016-05-17 21:29:38,795 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:38,796 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 04769812DEE54F748B8E42A563BF2E88
2016-05-17 21:29:38,796 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:38,796 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:38,796 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B20E9DB1ACA24B1488523AE5E93400A2
2016-05-17 21:29:38,796 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B20E9DB1ACA24B1488523AE5E93400A2
2016-05-17 21:29:38,798 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:38,804 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:38,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 04769812DEE54F748B8E42A563BF2E88
2016-05-17 21:29:38,804 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BE25195FDF274D109A7160EB0AD4712D, ordinal: 1
2016-05-17 21:29:38,805 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE25195FDF274D109A7160EB0AD4712D, mem on start (mb), free: 110, total: 360, max: 672
2016-05-17 21:29:38,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:38,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:38,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:38,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:38,805 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:38,806 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:38,806 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:38,806 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:38,806 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:38,806 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:38,807 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 04769812DEE54F748B8E42A563BF2E88, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:38,807 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B20E9DB1ACA24B1488523AE5E93400A2, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:38,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:38,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B20E9DB1ACA24B1488523AE5E93400A2 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:38,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B20E9DB1ACA24B1488523AE5E93400A2
2016-05-17 21:29:38,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:38,808 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:38,808 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:38,808 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:38,808 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B20E9DB1ACA24B1488523AE5E93400A2 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:38,808 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:38,809 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:38,809 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:38,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B20E9DB1ACA24B1488523AE5E93400A2
2016-05-17 21:29:38,810 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:38,810 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520578535_0001_1_01_000000_0
2016-05-17 21:29:38,810 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520578535_0001_1_00_000000_0_10147, runDuration: 39191, ]
2016-05-17 21:29:38,814 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:38,815 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 3FBE13DAAB66455A86A646836DC5020C
2016-05-17 21:29:38,815 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_-1"], id: 04769812DEE54F748B8E42A563BF2E88
2016-05-17 21:29:38,815 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520578535_0001_1_00_000000_0_10147] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520578815, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:38,815 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B20E9DB1ACA24B1488523AE5E93400A2
2016-05-17 21:29:38,815 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_-1"]
2016-05-17 21:29:38,815 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:38,816 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B20E9DB1ACA24B1488523AE5E93400A2] #25 in 2ms
2016-05-17 21:29:38,816 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 3FBE13DAAB66455A86A646836DC5020C, for 1 inputs
2016-05-17 21:29:38,816 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B20E9DB1ACA24B1488523AE5E93400A2] #25, status:false, isInterrupted:false
2016-05-17 21:29:38,817 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:38,818 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:38,818 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:38,818 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:38,818 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:38,819 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:38,819 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B20E9DB1ACA24B1488523AE5E93400A2
2016-05-17 21:29:38,819 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:38,819 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BE25195FDF274D109A7160EB0AD4712D, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:38,820 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:38,833 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@7bc9d355; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@7f51b0f0
2016-05-17 21:29:38,837 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:38,837 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:38,837 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520578535_0001_1_01_000000_0
2016-05-17 21:29:38,837 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520578535_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:38,838 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520578535_0001_1_01_000000_0
2016-05-17 21:29:38,838 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE25195FDF274D109A7160EB0AD4712D, mem on close (mb), free: 317, total: 361, max: 672
2016-05-17 21:29:38,838 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B20E9DB1ACA24B1488523AE5E93400A2
2016-05-17 21:29:38,839 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:38,840 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:38,840 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:38,840 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:38,840 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:38,841 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:38 Completed running task attempt: attempt_1463520578535_0001_1_01_000000_0
2016-05-17 21:29:38,842 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520578535_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:38,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520578819, Process_Duration=19, Process_End_Time=1463520578838, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:38,842 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:38,842 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BE25195FDF274D109A7160EB0AD4712D, taskAttemptId=attempt_1463520578535_0001_1_01_000000_0, startTime=1463520578788, finishTime=1463520578842, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520578819, Process_Duration=19, Process_End_Time=1463520578838, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:38,843 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520578535_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:38,843 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520578535_0001_1_01_000000_0
2016-05-17 21:29:38,843 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:TASK_FINISHED]: vertexName=BE25195FDF274D109A7160EB0AD4712D, taskId=task_1463520578535_0001_1_01_000000, startTime=1463520578788, finishTime=1463520578843, timeTaken=55, status=SUCCEEDED, successfulAttemptID=attempt_1463520578535_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520578819, Process_Duration=19, Process_End_Time=1463520578838, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:38,843 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520578535_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:38,843 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D] : 1
2016-05-17 21:29:38,843 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D]
2016-05-17 21:29:38,844 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:VERTEX_FINISHED]: vertexName=BE25195FDF274D109A7160EB0AD4712D, vertexId=vertex_1463520578535_0001_1_01, initRequestedTime=1463520578704, initedTime=1463520578711, startRequestedTime=1463520578717, startedTime=1463520578717, finishTime=1463520578844, timeTaken=127, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520578819, Process_Duration=19, Process_End_Time=1463520578838, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520578788, firstTasksToStart=[ task_1463520578535_0001_1_01_000000 ], lastTaskFinishTime=1463520578842, lastTasksToFinish=[ task_1463520578535_0001_1_01_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520578535_0001_1_01_000000 ], longestDurationTasks=[ task_1463520578535_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:38,844 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:38,842 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:38,844 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:38,844 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:38,845 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520578535_0001_1
2016-05-17 21:29:38,845 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520578535_0001_1
2016-05-17 21:29:38,845 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520578535_0001_1_00 [B20E9DB1ACA24B1488523AE5E93400A2]
2016-05-17 21:29:38,845 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 04769812DEE54F748B8E42A563BF2E88 for vertex: vertex_1463520578535_0001_1_01 [BE25195FDF274D109A7160EB0AD4712D]
2016-05-17 21:29:38,843 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:38,846 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:38,843 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:38,845 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520578535_0001_00_000002
2016-05-17 21:29:38,853 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520578535_0001_1, queueSize=0
2016-05-17 21:29:38,854 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:38,855 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520578535_0001_1, startTime=1463520578702, finishTime=1463520578852, timeTaken=150, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=345, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=756023296, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041157581, Process_Duration=21, Process_End_Time=2927041157602, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:38,855 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520578535_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:38,855 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520578535_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:38,855 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520578535_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:38,855 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:38,855 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520578535_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520578535_0001_00_000002, stoppedTime=1463520578855, exitStatus=0
2016-05-17 21:29:38,856 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520578535_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:38 Completed Dag: dag_1463520578535_0001_1
2016-05-17 21:29:38,856 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520578535_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:38,856 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520578535_0001_1_post
2016-05-17 21:29:38,856 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:38,857 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:38,857 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520578535_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:38,857 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520578535_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:38,857 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520578535_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:38,858 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520578535_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:38,858 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520578535_0001_00_000002, Exiting
2016-05-17 21:29:38,858 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520578535_0001_00_000002
2016-05-17 21:29:38,858 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520578535_0001_00_000002
2016-05-17 21:29:38,858 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520578535_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:38,858 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520578535_0001_00_000002 completed successfully
2016-05-17 21:29:38,860 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520578535_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:38,862 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/0_UNKNOWN_-1, applicationId=application_1463520578535_0001
2016-05-17 21:29:38,863 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:38,863 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:38,864 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:38,864 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:38,864 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:38,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:38,864 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:38,864 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:38,864 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:38,864 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:38,865 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:38,865 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:38,865 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:38,869 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/36C11CE86D2C4F2FB1DA6ABEC41327D0/ECC8B53579F045B8A868571EB73C599C/.tez/application_1463520578535_0001
2016-05-17 21:29:38,869 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_-1/_temporary
2016-05-17 21:29:38,870 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.352
2016-05-17 21:29:38,930 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_-1
2016-05-17 21:29:38,949 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:38,949 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:38,974 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 21:29:38,976 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-17 21:29:38,976 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:38,976 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:38,976 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_'field'
2016-05-17 21:29:38,995 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:38,995 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:39,011 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:39,011 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:39,023 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 21:29:39,034 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 21:29:39,035 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:39,035 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:39,035 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:39,048 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:39,053 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_RESULTS
2016-05-17 21:29:39,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:39,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:39,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_RESULTS"]
2016-05-17 21:29:39,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:39,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:39,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:39,095 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...latform/0_UNKNOWN_RESULTS
2016-05-17 21:29:39,095 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:39,111 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:39,115 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:39,116 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:39,116 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:39,124 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CD3A999FE24247CFB1B04F64CF399489/AB2840BA005F4191866069CBD060B06E/.tez/application_1463520579115_0001 doesn't exist and is created
2016-05-17 21:29:39,139 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:39,139 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:39,140 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CD3A999FE24247CFB1B04F64CF399489/AB2840BA005F4191866069CBD060B06E/.tez/application_1463520579115_0001
2016-05-17 21:29:39,142 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520579115_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:39,151 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:39,151 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:39,152 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:39,153 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:39,153 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CD3A999FE24247CFB1B04F64CF399489/AB2840BA005F4191866069CBD060B06E/.tez/application_1463520579115_0001/localmode-log-dir
2016-05-17 21:29:39,153 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CD3A999FE24247CFB1B04F64CF399489/AB2840BA005F4191866069CBD060B06E/.tez/application_1463520579115_0001/localmode-log-dir/history.txt.appattempt_1463520579115_0001_000000, maxErrors=10
2016-05-17 21:29:39,154 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:39,154 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520579115_0001_000000, appSubmitTime=1463520579142, launchTime=1463520579142
2016-05-17 21:29:39,160 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:39,160 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:39,161 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:39,161 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:39,162 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48414
2016-05-17 21:29:39,165 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:39,167 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520579115_0001_000000, startTime=1463520579167
2016-05-17 21:29:39,167 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:39,167 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:39,240 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:39,240 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:39,240 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_RESULTS, applicationId=application_1463520579115_0001, dagName=[CD3A999FE24247CFB1B04F64CF399489/AB2840BA005F4191866069CBD060B06E] every/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-05-17 21:29:39,254 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [CD3A999FE24247CFB1B04F64CF399489/AB2840BA005F4191866069CBD060B06E] every/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-05-17 21:29:39,255 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520579115_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CD3A999FE24247CFB1B04F64CF399489/AB2840BA005F4191866069CBD060B06E/.tez/application_1463520579115_0001/localmode-log-dir/dag_1463520579115_0001_1.dot
2016-05-17 21:29:39,256 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CD3A999FE24247CFB1B04F64CF399489/AB2840BA005F4191866069CBD060B06E/.tez/application_1463520579115_0001/localmode-log-dir/dag_1463520579115_0001_1-tez-dag.pb.txt
2016-05-17 21:29:39,262 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520579115_0001_1
2016-05-17 21:29:39,262 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:39,262 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [CD3A999FE24247CFB1B04F64CF399489/AB2840BA005F4191866069CBD060B06E] every/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-05-17 21:29:39 Running Dag: dag_1463520579115_0001_1
2016-05-17 21:29:39,273 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520579115_0001_1, submitTime=1463520579254
2016-05-17 21:29:39,273 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:39,274 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex ACA749C7BDB84AA2BC7537242BD54A31
2016-05-17 21:29:39,274 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:39,275 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520579115_0001_1, initTime=1463520579273
2016-05-17 21:29:39,275 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:39,276 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520579115_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:39,276 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:DAG_STARTED]: dagID=dag_1463520579115_0001_1, startTime=1463520579276
2016-05-17 21:29:39,276 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:39,276 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:39,276 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520579115_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:39,276 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EED10131E8554B82BCA8C6B4F36A1C20 : {0AEF9C3585C24E3EAE739F87DD62F8EE={InputName=0AEF9C3585C24E3EAE739F87DD62F8EE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:39,276 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0AEF9C3585C24E3EAE739F87DD62F8EE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:39,277 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20]
2016-05-17 21:29:39,277 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20]
2016-05-17 21:29:39,277 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20]
2016-05-17 21:29:39,277 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20]. Starting root input initializers: 1
2016-05-17 21:29:39,277 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:39,277 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31]
2016-05-17 21:29:39,278 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_RESULTS, applicationId=application_1463520579115_0001, dagName=[CD3A999FE24247CFB1B04F64CF399489/AB2840BA005F4191866069CBD060B06E] every/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-05-17 21:29:39,280 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:39,280 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31]
2016-05-17 21:29:39,280 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31]
2016-05-17 21:29:39,280 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31]
2016-05-17 21:29:39,280 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=EB42D23E4DD94B91B85A8FE742975005, vertexId=vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:39,281 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=EB42D23E4DD94B91B85A8FE742975005, vertexId=vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31]
2016-05-17 21:29:39,282 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=EB42D23E4DD94B91B85A8FE742975005, vertexName=ACA749C7BDB84AA2BC7537242BD54A31, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:39,283 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=EB42D23E4DD94B91B85A8FE742975005, vertexId=vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31]
2016-05-17 21:29:39,282 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0AEF9C3585C24E3EAE739F87DD62F8EE on vertex vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20]
2016-05-17 21:29:39,285 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:39,285 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ACA749C7BDB84AA2BC7537242BD54A31, vertexId=vertex_1463520579115_0001_1_01, initRequestedTime=1463520579277, initedTime=1463520579284, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:39,285 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:39,285 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:39,286 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0AEF9C3585C24E3EAE739F87DD62F8EE asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:39,286 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:39,286 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:39,287 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:39,288 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0AEF9C3585C24E3EAE739F87DD62F8EE on vertex vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20]
2016-05-17 21:29:39,288 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:39,288 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0AEF9C3585C24E3EAE739F87DD62F8EE=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:39,288 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20] parallelism set to 1
2016-05-17 21:29:39,289 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:39,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520579115_0001_1_00, eventCount=1
2016-05-17 21:29:39,289 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:39,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EED10131E8554B82BCA8C6B4F36A1C20, vertexId=vertex_1463520579115_0001_1_00, initRequestedTime=1463520579276, initedTime=1463520579289, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:39,290 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20] with distanceFromRoot: 0
2016-05-17 21:29:39,290 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:39,290 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EED10131E8554B82BCA8C6B4F36A1C20
2016-05-17 21:29:39,290 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20]
2016-05-17 21:29:39,290 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:39,291 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520579115_0001_1_00, startRequestedTime=1463520579290, startedTime=1463520579290
2016-05-17 21:29:39,291 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:39,291 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:39,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:TASK_STARTED]: vertexName=EED10131E8554B82BCA8C6B4F36A1C20, taskId=task_1463520579115_0001_1_00_000000, scheduledTime=1463520579291, launchTime=1463520579291
2016-05-17 21:29:39,292 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520579115_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:39,292 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520579115_0001_1_00 for vertex: vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31] numStartedSources: 1 numSources: 1
2016-05-17 21:29:39,292 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31] with distanceFromRoot: 1
2016-05-17 21:29:39,292 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20]
2016-05-17 21:29:39,293 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: EED10131E8554B82BCA8C6B4F36A1C20 in vertex: ACA749C7BDB84AA2BC7537242BD54A31
2016-05-17 21:29:39,293 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: ACA749C7BDB84AA2BC7537242BD54A31 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:39,293 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520579115_0001, with dag id: dag_1463520579115_0001_1
2016-05-17 21:29:39,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:39,294 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520579115_0001_1_01, startRequestedTime=1463520579293, startedTime=1463520579293
2016-05-17 21:29:39,294 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:39,294 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [CD3A999FE24247CFB1B04F64CF399489/AB2840BA005F4191866069CBD060B06E] every/(1/1) ...latform/0_UNKNOWN_RESULTS, VertexName: EED10131E8554B82BCA8C6B4F36A1C20, VertexParallelism: 1, TaskAttemptID:attempt_1463520579115_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0AEF9C3585C24E3EAE739F87DD62F8EE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=ACA749C7BDB84AA2BC7537242BD54A31, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:39,294 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520579115_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:39,294 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:39,295 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:39,295 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:39,295 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:39,296 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520579115_0001_00_000001
2016-05-17 21:29:39,296 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579115_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:39,297 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520579115_0001_00_000001
2016-05-17 21:29:39,298 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:39,299 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579115_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:39,299 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:39,299 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520579115_0001_00_000001, launchTime=1463520579299
2016-05-17 21:29:39,299 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520579115_0001_00_000001
2016-05-17 21:29:39,300 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520579115_0001_1_00_000000_0] to container: [container_1463520579115_0001_00_000001]
2016-05-17 21:29:39,300 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579115_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:39,301 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520579115_0001_1_00_000000_0] started. Is using containerId: [container_1463520579115_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:39,301 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EED10131E8554B82BCA8C6B4F36A1C20, taskAttemptId=attempt_1463520579115_0001_1_00_000000_0, startTime=1463520579300, containerId=container_1463520579115_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520579115_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:39,301 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:39,301 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520579115_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:39,301 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520579115_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:39,301 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520579115_0001_00_000001 given task: attempt_1463520579115_0001_1_00_000000_0
2016-05-17 21:29:39,301 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520579115_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520579115_0001_1_00_000000_0
2016-05-17 21:29:39 Starting to run new task attempt: attempt_1463520579115_0001_1_00_000000_0
2016-05-17 21:29:39,302 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520579115_0001_1_00_000000_0
2016-05-17 21:29:39,302 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:39,302 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:39,302 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:39,302 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:39,302 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:39,302 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:39,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CD3A999FE24247CFB1B04F64CF399489/AB2840BA005F4191866069CBD060B06E] every/(1/1) ...latform/0_UNKNOWN_RESULTS, VertexName: EED10131E8554B82BCA8C6B4F36A1C20, VertexParallelism: 1, TaskAttemptID:attempt_1463520579115_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0AEF9C3585C24E3EAE739F87DD62F8EE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=ACA749C7BDB84AA2BC7537242BD54A31, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:39,303 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:39,303 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520579115_0001_1_00_000000_0
2016-05-17 21:29:39,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:39,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:39,303 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520579115_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:39,304 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:39,304 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=ACA749C7BDB84AA2BC7537242BD54A31, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:39,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:39,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: ACA749C7BDB84AA2BC7537242BD54A31
2016-05-17 21:29:39,304 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0AEF9C3585C24E3EAE739F87DD62F8EE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:39,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:39,306 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:39,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0AEF9C3585C24E3EAE739F87DD62F8EE
2016-05-17 21:29:39,306 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0AEF9C3585C24E3EAE739F87DD62F8EE
2016-05-17 21:29:39,307 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:39,307 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: ACA749C7BDB84AA2BC7537242BD54A31
2016-05-17 21:29:39,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: ACA749C7BDB84AA2BC7537242BD54A31
2016-05-17 21:29:39,309 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:39,309 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0AEF9C3585C24E3EAE739F87DD62F8EE
2016-05-17 21:29:39,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0AEF9C3585C24E3EAE739F87DD62F8EE
2016-05-17 21:29:39,311 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EED10131E8554B82BCA8C6B4F36A1C20, ordinal: 0
2016-05-17 21:29:39,311 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EED10131E8554B82BCA8C6B4F36A1C20, mem on start (mb), free: 211, total: 361, max: 672
2016-05-17 21:29:39,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:39,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:39,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:39,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:39,312 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:39,312 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:39,312 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:39,312 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:39,313 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:39,313 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:39,313 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0AEF9C3585C24E3EAE739F87DD62F8EE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:39,313 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, ACA749C7BDB84AA2BC7537242BD54A31, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:39,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:39,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0AEF9C3585C24E3EAE739F87DD62F8EE being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:39,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:39,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:39,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0AEF9C3585C24E3EAE739F87DD62F8EE
2016-05-17 21:29:39,314 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:39,315 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0AEF9C3585C24E3EAE739F87DD62F8EE
2016-05-17 21:29:39,315 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:39,315 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520579115_0001_1_00_000000_0
2016-05-17 21:29:39,315 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:39,316 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@42ee28bf
2016-05-17 21:29:39,316 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@42ee28bf
2016-05-17 21:29:39,317 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:39,317 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:39,318 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:39,318 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0AEF9C3585C24E3EAE739F87DD62F8EE
2016-05-17 21:29:39,319 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: DB8E42C453334EEEB8D21304C9279F5D
2016-05-17 21:29:39,319 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] DB8E42C453334EEEB8D21304C9279F5D
2016-05-17 21:29:39,320 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:39,321 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:39,322 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@4293e735; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@7c306ab8; comparator=cascading.tuple.hadoop.util.TupleComparator@4eae95ba; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:39,322 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:39,322 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:39,322 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:39,336 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:39,336 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:39,336 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:39,336 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:39,337 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:39,337 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:39,337 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:39,338 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EED10131E8554B82BCA8C6B4F36A1C20, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:39,342 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520579115_0001_1_00_000000_0
2016-05-17 21:29:39,342 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EED10131E8554B82BCA8C6B4F36A1C20, mem on close (mb), free: 109, total: 361, max: 672
2016-05-17 21:29:39,342 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:39,342 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:39,343 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:39,343 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:39,347 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:39 Completed running task attempt: attempt_1463520579115_0001_1_00_000000_0
2016-05-17 21:29:39,350 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520579115_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:39,350 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520579115_0001_1_00, eventCount=1
2016-05-17 21:29:39,350 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EED10131E8554B82BCA8C6B4F36A1C20, taskAttemptId=attempt_1463520579115_0001_1_00_000000_0, startTime=1463520579300, finishTime=1463520579350, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520579338, Process_Duration=4, Process_End_Time=1463520579342, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:39,351 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520579115_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:39,351 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:39,351 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:39,351 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520579115_0001_1_00_000000_0
2016-05-17 21:29:39,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:39,352 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:TASK_FINISHED]: vertexName=EED10131E8554B82BCA8C6B4F36A1C20, taskId=task_1463520579115_0001_1_00_000000, startTime=1463520579300, finishTime=1463520579351, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1463520579115_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520579338, Process_Duration=4, Process_End_Time=1463520579342, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:39,352 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520579115_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:39,352 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20] : 1
2016-05-17 21:29:39,352 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20]
2016-05-17 21:29:39,353 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:39,353 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:VERTEX_FINISHED]: vertexName=EED10131E8554B82BCA8C6B4F36A1C20, vertexId=vertex_1463520579115_0001_1_00, initRequestedTime=1463520579276, initedTime=1463520579289, startRequestedTime=1463520579290, startedTime=1463520579290, finishTime=1463520579352, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520579338, Process_Duration=4, Process_End_Time=1463520579342, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520579300, firstTasksToStart=[ task_1463520579115_0001_1_00_000000 ], lastTaskFinishTime=1463520579350, lastTasksToFinish=[ task_1463520579115_0001_1_00_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520579115_0001_1_00_000000 ], longestDurationTasks=[ task_1463520579115_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:39,353 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:39,353 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579115_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:39,353 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520579338, Process_Duration=4, Process_End_Time=1463520579342, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:39,353 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520579115_0001_00_000001, stoppedTime=1463520579353, exitStatus=0
2016-05-17 21:29:39,354 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:39,354 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:39,353 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:39,354 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579115_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:39,354 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520579115_0001_00_000001
2016-05-17 21:29:39,354 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31] attempt: attempt_1463520579115_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:39,355 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520579115_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:39,355 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520579115_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:39,355 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520579115_0001_00_000001, Exiting
2016-05-17 21:29:39,355 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520579115_0001_00_000001
2016-05-17 21:29:39,355 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520579115_0001_00_000001
2016-05-17 21:29:39,355 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: ACA749C7BDB84AA2BC7537242BD54A31
2016-05-17 21:29:39,355 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31]
2016-05-17 21:29:39,356 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:39,356 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520579115_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:39,356 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520579115_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:39,357 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520579115_0001_00_000001 completed successfully
2016-05-17 21:29:39,357 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:39,356 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:TASK_STARTED]: vertexName=ACA749C7BDB84AA2BC7537242BD54A31, taskId=task_1463520579115_0001_1_01_000000, scheduledTime=1463520579356, launchTime=1463520579356
2016-05-17 21:29:39,357 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520579115_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:39,357 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579115_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:39,358 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31]
2016-05-17 21:29:39,358 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579115_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:39,358 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [CD3A999FE24247CFB1B04F64CF399489/AB2840BA005F4191866069CBD060B06E] every/(1/1) ...latform/0_UNKNOWN_RESULTS, VertexName: ACA749C7BDB84AA2BC7537242BD54A31, VertexParallelism: 1, TaskAttemptID:attempt_1463520579115_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EED10131E8554B82BCA8C6B4F36A1C20, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=EB42D23E4DD94B91B85A8FE742975005, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:39,358 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520579115_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:39,358 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:39,359 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520579115_0001_00_000002
2016-05-17 21:29:39,359 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579115_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:39,360 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520579115_0001_00_000002
2016-05-17 21:29:39,361 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:39,363 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579115_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:39,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520579115_0001_00_000002, launchTime=1463520579363
2016-05-17 21:29:39,363 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:39,364 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520579115_0001_00_000002
2016-05-17 21:29:39,364 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520579115_0001_1_01_000000_0] to container: [container_1463520579115_0001_00_000002]
2016-05-17 21:29:39,365 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579115_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:39,365 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520579115_0001_1_01_000000_0] started. Is using containerId: [container_1463520579115_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:39,365 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ACA749C7BDB84AA2BC7537242BD54A31, taskAttemptId=attempt_1463520579115_0001_1_01_000000_0, startTime=1463520579365, containerId=container_1463520579115_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520579115_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:39,365 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520579115_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:39,365 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520579115_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:39,366 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:39,366 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520579115_0001_00_000002 given task: attempt_1463520579115_0001_1_01_000000_0
2016-05-17 21:29:39,366 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520579115_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520579115_0001_1_01_000000_0
2016-05-17 21:29:39 Starting to run new task attempt: attempt_1463520579115_0001_1_01_000000_0
2016-05-17 21:29:39,366 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520579115_0001_1_01_000000_0
2016-05-17 21:29:39,366 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:39,366 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:39,367 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:39,367 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:39,367 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:39,367 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:39,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CD3A999FE24247CFB1B04F64CF399489/AB2840BA005F4191866069CBD060B06E] every/(1/1) ...latform/0_UNKNOWN_RESULTS, VertexName: ACA749C7BDB84AA2BC7537242BD54A31, VertexParallelism: 1, TaskAttemptID:attempt_1463520579115_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EED10131E8554B82BCA8C6B4F36A1C20, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=EB42D23E4DD94B91B85A8FE742975005, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:39,368 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:39,368 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520579115_0001_1_01_000000_0
2016-05-17 21:29:39,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:39,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:39,368 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520579115_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:39,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EB42D23E4DD94B91B85A8FE742975005, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:39,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:39,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:39,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EED10131E8554B82BCA8C6B4F36A1C20, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:39,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:39,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EED10131E8554B82BCA8C6B4F36A1C20
2016-05-17 21:29:39,373 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:39,373 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:39,373 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: EED10131E8554B82BCA8C6B4F36A1C20
2016-05-17 21:29:39,383 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EED10131E8554B82BCA8C6B4F36A1C20
2016-05-17 21:29:39,384 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:39,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EB42D23E4DD94B91B85A8FE742975005
2016-05-17 21:29:39,386 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:39,386 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: EB42D23E4DD94B91B85A8FE742975005
2016-05-17 21:29:39,389 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ACA749C7BDB84AA2BC7537242BD54A31, ordinal: 1
2016-05-17 21:29:39,389 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ACA749C7BDB84AA2BC7537242BD54A31, mem on start (mb), free: 315, total: 361, max: 672
2016-05-17 21:29:39,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:39,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:39,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:39,393 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:39,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EB42D23E4DD94B91B85A8FE742975005
2016-05-17 21:29:39,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:39,393 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:39,393 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:39,393 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:39,393 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:39,394 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:39,394 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:39,394 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EED10131E8554B82BCA8C6B4F36A1C20, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:39,394 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EB42D23E4DD94B91B85A8FE742975005, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:39,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:39,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EED10131E8554B82BCA8C6B4F36A1C20 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:39,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EED10131E8554B82BCA8C6B4F36A1C20
2016-05-17 21:29:39,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:39,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:39,395 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:39,396 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:39,396 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: EED10131E8554B82BCA8C6B4F36A1C20 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:39,396 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:39,396 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:39,396 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:39,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EED10131E8554B82BCA8C6B4F36A1C20
2016-05-17 21:29:39,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:39,398 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520579115_0001_1_01_000000_0
2016-05-17 21:29:39,399 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520579115_0001_1_00_000000_0_10152, runDuration: 43231, ]
2016-05-17 21:29:39,400 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:39,401 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520579115_0001_1_00_000000_0_10152] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520579401, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:39,401 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : EED10131E8554B82BCA8C6B4F36A1C20
2016-05-17 21:29:39,401 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: DB8E42C453334EEEB8D21304C9279F5D
2016-05-17 21:29:39,401 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_RESULTS"], id: EB42D23E4DD94B91B85A8FE742975005
2016-05-17 21:29:39,402 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_RESULTS"]
2016-05-17 21:29:39,402 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] DB8E42C453334EEEB8D21304C9279F5D, for 1 inputs
2016-05-17 21:29:39,402 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:39,402 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [EED10131E8554B82BCA8C6B4F36A1C20] #26 in 2ms
2016-05-17 21:29:39,402 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [EED10131E8554B82BCA8C6B4F36A1C20] #26, status:false, isInterrupted:false
2016-05-17 21:29:39,403 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:39,404 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:39,404 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:39,404 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:39,404 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:39,404 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:39,405 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : EED10131E8554B82BCA8C6B4F36A1C20
2016-05-17 21:29:39,405 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:39,405 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ACA749C7BDB84AA2BC7537242BD54A31, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:39,406 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:39,407 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@51ebde29; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@54467fc3
2016-05-17 21:29:39,411 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:39,411 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:39,411 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520579115_0001_1_01_000000_0
2016-05-17 21:29:39,411 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520579115_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:39,412 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520579115_0001_1_01_000000_0
2016-05-17 21:29:39,412 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ACA749C7BDB84AA2BC7537242BD54A31, mem on close (mb), free: 312, total: 361, max: 672
2016-05-17 21:29:39,412 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: EED10131E8554B82BCA8C6B4F36A1C20
2016-05-17 21:29:39,412 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:39,413 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:39,413 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:39,413 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:39,413 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:39,414 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:39 Completed running task attempt: attempt_1463520579115_0001_1_01_000000_0
2016-05-17 21:29:39,414 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520579115_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:39,415 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520579405, Process_Duration=7, Process_End_Time=1463520579412, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:39,415 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:39,415 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:39,415 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ACA749C7BDB84AA2BC7537242BD54A31, taskAttemptId=attempt_1463520579115_0001_1_01_000000_0, startTime=1463520579365, finishTime=1463520579415, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520579405, Process_Duration=7, Process_End_Time=1463520579412, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:39,415 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520579115_0001_00_000002
2016-05-17 21:29:39,415 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520579115_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:39,415 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:39,415 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520579115_0001_1_01_000000_0
2016-05-17 21:29:39,416 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:39,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:TASK_FINISHED]: vertexName=ACA749C7BDB84AA2BC7537242BD54A31, taskId=task_1463520579115_0001_1_01_000000, startTime=1463520579365, finishTime=1463520579416, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1463520579115_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520579405, Process_Duration=7, Process_End_Time=1463520579412, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:39,416 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520579115_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:39,416 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31] : 1
2016-05-17 21:29:39,416 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31]
2016-05-17 21:29:39,417 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:VERTEX_FINISHED]: vertexName=ACA749C7BDB84AA2BC7537242BD54A31, vertexId=vertex_1463520579115_0001_1_01, initRequestedTime=1463520579277, initedTime=1463520579284, startRequestedTime=1463520579293, startedTime=1463520579293, finishTime=1463520579417, timeTaken=124, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520579405, Process_Duration=7, Process_End_Time=1463520579412, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520579365, firstTasksToStart=[ task_1463520579115_0001_1_01_000000 ], lastTaskFinishTime=1463520579415, lastTasksToFinish=[ task_1463520579115_0001_1_01_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520579115_0001_1_01_000000 ], longestDurationTasks=[ task_1463520579115_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:39,417 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:39,418 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579115_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:39,418 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520579115_0001_00_000002, stoppedTime=1463520579418, exitStatus=0
2016-05-17 21:29:39,418 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579115_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:39,416 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:39,419 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:39,419 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:39,420 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:39,420 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520579115_0001_1
2016-05-17 21:29:39,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520579115_0001_1
2016-05-17 21:29:39,421 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: EB42D23E4DD94B91B85A8FE742975005 for vertex: vertex_1463520579115_0001_1_01 [ACA749C7BDB84AA2BC7537242BD54A31]
2016-05-17 21:29:39,425 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520579115_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:39,426 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520579115_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:39,426 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520579115_0001_00_000002, Exiting
2016-05-17 21:29:39,426 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520579115_0001_00_000002
2016-05-17 21:29:39,426 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520579115_0001_00_000002
2016-05-17 21:29:39,426 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520579115_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:39,426 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520579115_0001_00_000002 completed successfully
2016-05-17 21:29:39,428 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520579115_0001_1_00 [EED10131E8554B82BCA8C6B4F36A1C20]
2016-05-17 21:29:39,429 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520579115_0001_1, queueSize=0
2016-05-17 21:29:39,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:39,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579115_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520579115_0001_1, startTime=1463520579276, finishTime=1463520579428, timeTaken=152, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=345, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=757071872, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041158743, Process_Duration=11, Process_End_Time=2927041158754, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:39,431 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520579115_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:39,431 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520579115_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:39,431 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579115_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:39,431 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520579115_0001_00_000002
2016-05-17 21:29:39 Completed Dag: dag_1463520579115_0001_1
2016-05-17 21:29:39,432 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520579115_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:39,432 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520579115_0001_1_post
2016-05-17 21:29:39,432 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:39,432 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:39,437 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_UNKNOWN_RESULTS, applicationId=application_1463520579115_0001
2016-05-17 21:29:39,439 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:39,439 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:39,439 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:39,439 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:39,439 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:39,439 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:39,439 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:39,440 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:39,440 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:39,440 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:39,440 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:39,441 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:39,441 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:39,445 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CD3A999FE24247CFB1B04F64CF399489/AB2840BA005F4191866069CBD060B06E/.tez/application_1463520579115_0001
2016-05-17 21:29:39,445 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_RESULTS/_temporary
2016-05-17 21:29:39,445 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.351
2016-05-17 21:29:39,512 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_RESULTS
2016-05-17 21:29:39,535 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:39,535 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:39,578 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 21:29:39,586 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 21:29:39,586 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:39,586 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:39,587 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:39,603 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:39,608 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_ALL
2016-05-17 21:29:39,651 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:39,651 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:39,651 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_ALL"]
2016-05-17 21:29:39,652 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:39,652 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:39,652 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:39,652 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...blyplatform/0_UNKNOWN_ALL
2016-05-17 21:29:39,653 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:39,664 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:39,668 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:39,669 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:39,669 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:39,680 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B68253108FB448778DE78A6B0F1AD518/CCE0AB81C6044CA5A76D5202408BD26C/.tez/application_1463520579668_0001 doesn't exist and is created
2016-05-17 21:29:39,696 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:39,696 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:39,697 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B68253108FB448778DE78A6B0F1AD518/CCE0AB81C6044CA5A76D5202408BD26C/.tez/application_1463520579668_0001
2016-05-17 21:29:39,699 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520579668_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:39,708 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:39,709 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:39,711 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:39,711 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:39,711 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B68253108FB448778DE78A6B0F1AD518/CCE0AB81C6044CA5A76D5202408BD26C/.tez/application_1463520579668_0001/localmode-log-dir
2016-05-17 21:29:39,712 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B68253108FB448778DE78A6B0F1AD518/CCE0AB81C6044CA5A76D5202408BD26C/.tez/application_1463520579668_0001/localmode-log-dir/history.txt.appattempt_1463520579668_0001_000000, maxErrors=10
2016-05-17 21:29:39,713 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:39,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520579668_0001_000000, appSubmitTime=1463520579699, launchTime=1463520579699
2016-05-17 21:29:39,718 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:39,718 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:42762
2016-05-17 21:29:39,722 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:39,723 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:39,723 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:39,724 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:39,725 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520579668_0001_000000, startTime=1463520579725
2016-05-17 21:29:39,725 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:39,725 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:39,787 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:39,788 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:39,796 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:39,797 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:39,797 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/0_UNKNOWN_ALL, applicationId=application_1463520579668_0001, dagName=[B68253108FB448778DE78A6B0F1AD518/CCE0AB81C6044CA5A76D5202408BD26C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-05-17 21:29:39,810 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [B68253108FB448778DE78A6B0F1AD518/CCE0AB81C6044CA5A76D5202408BD26C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-05-17 21:29:39,811 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520579668_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B68253108FB448778DE78A6B0F1AD518/CCE0AB81C6044CA5A76D5202408BD26C/.tez/application_1463520579668_0001/localmode-log-dir/dag_1463520579668_0001_1.dot
2016-05-17 21:29:39,812 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B68253108FB448778DE78A6B0F1AD518/CCE0AB81C6044CA5A76D5202408BD26C/.tez/application_1463520579668_0001/localmode-log-dir/dag_1463520579668_0001_1-tez-dag.pb.txt
2016-05-17 21:29:39,818 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520579668_0001_1
2016-05-17 21:29:39,818 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:39,818 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [B68253108FB448778DE78A6B0F1AD518/CCE0AB81C6044CA5A76D5202408BD26C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-05-17 21:29:39 Running Dag: dag_1463520579668_0001_1
2016-05-17 21:29:39,828 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520579668_0001_1, submitTime=1463520579810
2016-05-17 21:29:39,828 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:39,829 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex AE9F50BD7CB94B9794FFD82A805430B2
2016-05-17 21:29:39,829 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:39,829 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520579668_0001_1, initTime=1463520579828
2016-05-17 21:29:39,829 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:39,829 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520579668_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:39,830 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:DAG_STARTED]: dagID=dag_1463520579668_0001_1, startTime=1463520579829
2016-05-17 21:29:39,830 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:39,830 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:39,830 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520579668_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:39,830 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D903B4CBCC1749C789EA91097CCB0A85 : {4720AA00025C410DA626793F290321F2={InputName=4720AA00025C410DA626793F290321F2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:39,830 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4720AA00025C410DA626793F290321F2, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:39,830 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85]
2016-05-17 21:29:39,830 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85]
2016-05-17 21:29:39,830 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85]
2016-05-17 21:29:39,831 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85]. Starting root input initializers: 1
2016-05-17 21:29:39,831 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:39,831 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2]
2016-05-17 21:29:39,831 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4720AA00025C410DA626793F290321F2 on vertex vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85]
2016-05-17 21:29:39,833 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:39,833 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/0_UNKNOWN_ALL, applicationId=application_1463520579668_0001, dagName=[B68253108FB448778DE78A6B0F1AD518/CCE0AB81C6044CA5A76D5202408BD26C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-05-17 21:29:39,833 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4720AA00025C410DA626793F290321F2 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:39,833 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:39,834 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:39,834 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:39,834 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2]
2016-05-17 21:29:39,834 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2]
2016-05-17 21:29:39,834 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2]
2016-05-17 21:29:39,834 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F4D8C8056A184D43958C958556BB3880, vertexId=vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:39,835 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F4D8C8056A184D43958C958556BB3880, vertexId=vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2]
2016-05-17 21:29:39,835 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:39,835 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4720AA00025C410DA626793F290321F2 on vertex vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85]
2016-05-17 21:29:39,836 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F4D8C8056A184D43958C958556BB3880, vertexName=AE9F50BD7CB94B9794FFD82A805430B2, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:39,836 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F4D8C8056A184D43958C958556BB3880, vertexId=vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2]
2016-05-17 21:29:39,837 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AE9F50BD7CB94B9794FFD82A805430B2, vertexId=vertex_1463520579668_0001_1_01, initRequestedTime=1463520579831, initedTime=1463520579837, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:39,838 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:39,838 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:39,838 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4720AA00025C410DA626793F290321F2=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:39,838 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85] parallelism set to 1
2016-05-17 21:29:39,838 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:39,838 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:39,839 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520579668_0001_1_00, eventCount=1
2016-05-17 21:29:39,839 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D903B4CBCC1749C789EA91097CCB0A85, vertexId=vertex_1463520579668_0001_1_00, initRequestedTime=1463520579830, initedTime=1463520579839, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:39,840 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85] with distanceFromRoot: 0
2016-05-17 21:29:39,840 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:39,840 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D903B4CBCC1749C789EA91097CCB0A85
2016-05-17 21:29:39,840 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85]
2016-05-17 21:29:39,840 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:39,841 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520579668_0001_1_00, startRequestedTime=1463520579840, startedTime=1463520579840
2016-05-17 21:29:39,841 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:39,841 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:39,842 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:TASK_STARTED]: vertexName=D903B4CBCC1749C789EA91097CCB0A85, taskId=task_1463520579668_0001_1_00_000000, scheduledTime=1463520579842, launchTime=1463520579842
2016-05-17 21:29:39,842 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520579668_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:39,843 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520579668_0001_1_00 for vertex: vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2] numStartedSources: 1 numSources: 1
2016-05-17 21:29:39,843 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2] with distanceFromRoot: 1
2016-05-17 21:29:39,843 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85]
2016-05-17 21:29:39,843 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D903B4CBCC1749C789EA91097CCB0A85 in vertex: AE9F50BD7CB94B9794FFD82A805430B2
2016-05-17 21:29:39,844 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: AE9F50BD7CB94B9794FFD82A805430B2 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:39,842 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:39,844 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520579668_0001_1_01, startRequestedTime=1463520579843, startedTime=1463520579843
2016-05-17 21:29:39,845 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:39,845 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B68253108FB448778DE78A6B0F1AD518/CCE0AB81C6044CA5A76D5202408BD26C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL, VertexName: D903B4CBCC1749C789EA91097CCB0A85, VertexParallelism: 1, TaskAttemptID:attempt_1463520579668_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4720AA00025C410DA626793F290321F2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AE9F50BD7CB94B9794FFD82A805430B2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:39,845 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520579668_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:39,844 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:39,845 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:39,846 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:39,846 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:39,846 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:39,847 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520579668_0001_00_000001
2016-05-17 21:29:39,847 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579668_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:39,848 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520579668_0001_00_000001
2016-05-17 21:29:39,848 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:39,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:39,849 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520579668_0001_00_000001, launchTime=1463520579849
2016-05-17 21:29:39,850 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579668_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:39,850 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520579668_0001_00_000001
2016-05-17 21:29:39,851 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520579668_0001_1_00_000000_0] to container: [container_1463520579668_0001_00_000001]
2016-05-17 21:29:39,851 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579668_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:39,851 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520579668_0001, with dag id: dag_1463520579668_0001_1
2016-05-17 21:29:39,851 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520579668_0001_00_000001 given task: attempt_1463520579668_0001_1_00_000000_0
2016-05-17 21:29:39,851 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520579668_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520579668_0001_1_00_000000_0
2016-05-17 21:29:39,851 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520579668_0001_1_00_000000_0] started. Is using containerId: [container_1463520579668_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:39,852 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:39 Starting to run new task attempt: attempt_1463520579668_0001_1_00_000000_0
2016-05-17 21:29:39,852 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520579668_0001_1_00_000000_0
2016-05-17 21:29:39,852 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:39,852 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D903B4CBCC1749C789EA91097CCB0A85, taskAttemptId=attempt_1463520579668_0001_1_00_000000_0, startTime=1463520579851, containerId=container_1463520579668_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520579668_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:39,852 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520579668_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:39,852 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:39,853 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:39,853 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520579668_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:39,853 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:39,853 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:39,854 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:39,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B68253108FB448778DE78A6B0F1AD518/CCE0AB81C6044CA5A76D5202408BD26C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL, VertexName: D903B4CBCC1749C789EA91097CCB0A85, VertexParallelism: 1, TaskAttemptID:attempt_1463520579668_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4720AA00025C410DA626793F290321F2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AE9F50BD7CB94B9794FFD82A805430B2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:39,855 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:39,855 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520579668_0001_1_00_000000_0
2016-05-17 21:29:39,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:39,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:39,856 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520579668_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:39,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4720AA00025C410DA626793F290321F2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:39,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:39,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:39,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4720AA00025C410DA626793F290321F2
2016-05-17 21:29:39,857 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4720AA00025C410DA626793F290321F2
2016-05-17 21:29:39,859 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:39,859 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:39,859 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4720AA00025C410DA626793F290321F2
2016-05-17 21:29:39,859 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4720AA00025C410DA626793F290321F2
2016-05-17 21:29:39,861 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AE9F50BD7CB94B9794FFD82A805430B2, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:39,861 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:39,861 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D903B4CBCC1749C789EA91097CCB0A85, ordinal: 0
2016-05-17 21:29:39,862 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D903B4CBCC1749C789EA91097CCB0A85, mem on start (mb), free: 223, total: 361, max: 672
2016-05-17 21:29:39,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:39,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:39,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:39,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AE9F50BD7CB94B9794FFD82A805430B2
2016-05-17 21:29:39,864 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:39,864 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: AE9F50BD7CB94B9794FFD82A805430B2
2016-05-17 21:29:39,864 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AE9F50BD7CB94B9794FFD82A805430B2
2016-05-17 21:29:39,865 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:39,865 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:39,865 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:39,865 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:39,865 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:39,865 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:39,866 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:39,866 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4720AA00025C410DA626793F290321F2, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:39,866 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AE9F50BD7CB94B9794FFD82A805430B2, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:39,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:39,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4720AA00025C410DA626793F290321F2 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:39,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:39,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:39,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4720AA00025C410DA626793F290321F2
2016-05-17 21:29:39,867 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:39,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4720AA00025C410DA626793F290321F2
2016-05-17 21:29:39,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:39,867 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520579668_0001_1_00_000000_0
2016-05-17 21:29:39,870 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:39,870 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:39,870 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6eb4b976
2016-05-17 21:29:39,871 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 4720AA00025C410DA626793F290321F2
2016-05-17 21:29:39,871 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 3AC38C0BCAFD4A68B444839B903B8657
2016-05-17 21:29:39,871 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 3AC38C0BCAFD4A68B444839B903B8657
2016-05-17 21:29:39,871 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6eb4b976
2016-05-17 21:29:39,872 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:39,872 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:39,872 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:39,874 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:39,874 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@6acce3fe; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@318c6f79; comparator=cascading.tuple.hadoop.util.TupleComparator@7aea8a58; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:39,874 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:39,874 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:39,874 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:39,888 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:39,889 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:39,889 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:39,889 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:39,889 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:39,889 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:39,889 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:39,890 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D903B4CBCC1749C789EA91097CCB0A85, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:29:39,892 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520579668_0001_1_00_000000_0
2016-05-17 21:29:39,892 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D903B4CBCC1749C789EA91097CCB0A85, mem on close (mb), free: 121, total: 361, max: 672
2016-05-17 21:29:39,892 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:39,892 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:39,892 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:39,892 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:39,896 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:39 Completed running task attempt: attempt_1463520579668_0001_1_00_000000_0
2016-05-17 21:29:39,898 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520579668_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:39,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520579890, Process_Duration=2, Process_End_Time=1463520579892, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:39,899 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520579668_0001_1_00, eventCount=1
2016-05-17 21:29:39,899 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:39,899 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:39,899 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520579668_0001_00_000001
2016-05-17 21:29:39,899 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D903B4CBCC1749C789EA91097CCB0A85, taskAttemptId=attempt_1463520579668_0001_1_00_000000_0, startTime=1463520579851, finishTime=1463520579899, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520579890, Process_Duration=2, Process_End_Time=1463520579892, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:39,899 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520579668_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:39,899 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:39,900 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520579668_0001_1_00_000000_0
2016-05-17 21:29:39,900 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:39,900 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:39,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:TASK_FINISHED]: vertexName=D903B4CBCC1749C789EA91097CCB0A85, taskId=task_1463520579668_0001_1_00_000000, startTime=1463520579851, finishTime=1463520579900, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1463520579668_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520579890, Process_Duration=2, Process_End_Time=1463520579892, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:39,901 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520579668_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:39,901 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85] : 1
2016-05-17 21:29:39,901 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85]
2016-05-17 21:29:39,901 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:39,901 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:VERTEX_FINISHED]: vertexName=D903B4CBCC1749C789EA91097CCB0A85, vertexId=vertex_1463520579668_0001_1_00, initRequestedTime=1463520579830, initedTime=1463520579839, startRequestedTime=1463520579840, startedTime=1463520579840, finishTime=1463520579901, timeTaken=61, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520579890, Process_Duration=2, Process_End_Time=1463520579892, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520579851, firstTasksToStart=[ task_1463520579668_0001_1_00_000000 ], lastTaskFinishTime=1463520579899, lastTasksToFinish=[ task_1463520579668_0001_1_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520579668_0001_1_00_000000 ], longestDurationTasks=[ task_1463520579668_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:39,902 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:39,902 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579668_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:39,902 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520579668_0001_00_000001, stoppedTime=1463520579902, exitStatus=0
2016-05-17 21:29:39,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:39,902 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579668_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:39,903 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2] attempt: attempt_1463520579668_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:39,903 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: AE9F50BD7CB94B9794FFD82A805430B2
2016-05-17 21:29:39,903 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2]
2016-05-17 21:29:39,903 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:39,904 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520579668_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:39,904 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:39,904 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:TASK_STARTED]: vertexName=AE9F50BD7CB94B9794FFD82A805430B2, taskId=task_1463520579668_0001_1_01_000000, scheduledTime=1463520579904, launchTime=1463520579904
2016-05-17 21:29:39,905 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520579668_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:39,905 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579668_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:39,905 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2]
2016-05-17 21:29:39,905 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B68253108FB448778DE78A6B0F1AD518/CCE0AB81C6044CA5A76D5202408BD26C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL, VertexName: AE9F50BD7CB94B9794FFD82A805430B2, VertexParallelism: 1, TaskAttemptID:attempt_1463520579668_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D903B4CBCC1749C789EA91097CCB0A85, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F4D8C8056A184D43958C958556BB3880, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:39,906 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520579668_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:39,906 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:39,906 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520579668_0001_00_000002
2016-05-17 21:29:39,907 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579668_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:39,908 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520579668_0001_00_000002
2016-05-17 21:29:39,908 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:39,909 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:39,909 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520579668_0001_00_000002, launchTime=1463520579909
2016-05-17 21:29:39,909 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579668_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:39,909 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520579668_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:39,909 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520579668_0001_00_000002
2016-05-17 21:29:39,909 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520579668_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:39,910 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520579668_0001_00_000001, Exiting
2016-05-17 21:29:39,910 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520579668_0001_00_000001
2016-05-17 21:29:39,910 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520579668_0001_00_000001
2016-05-17 21:29:39,910 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520579668_0001_1_01_000000_0] to container: [container_1463520579668_0001_00_000002]
2016-05-17 21:29:39,910 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579668_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:39,910 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520579668_0001_00_000002 given task: attempt_1463520579668_0001_1_01_000000_0
2016-05-17 21:29:39,911 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520579668_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520579668_0001_1_01_000000_0
2016-05-17 21:29:39 Starting to run new task attempt: attempt_1463520579668_0001_1_01_000000_0
2016-05-17 21:29:39,911 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520579668_0001_1_01_000000_0] started. Is using containerId: [container_1463520579668_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:39,911 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520579668_0001_1_01_000000_0
2016-05-17 21:29:39,911 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:39,911 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:39,911 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:39,912 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:39,912 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520579668_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:39,912 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:39,912 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AE9F50BD7CB94B9794FFD82A805430B2, taskAttemptId=attempt_1463520579668_0001_1_01_000000_0, startTime=1463520579911, containerId=container_1463520579668_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520579668_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:39,912 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:39,912 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520579668_0001_00_000001 completed successfully
2016-05-17 21:29:39,912 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:39,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B68253108FB448778DE78A6B0F1AD518/CCE0AB81C6044CA5A76D5202408BD26C] every/(1/1) ...blyplatform/0_UNKNOWN_ALL, VertexName: AE9F50BD7CB94B9794FFD82A805430B2, VertexParallelism: 1, TaskAttemptID:attempt_1463520579668_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D903B4CBCC1749C789EA91097CCB0A85, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F4D8C8056A184D43958C958556BB3880, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:39,913 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:39,913 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520579668_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:39,913 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520579668_0001_1_01_000000_0
2016-05-17 21:29:39,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:39,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:39,913 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520579668_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:39,914 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579668_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:39,914 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520579668_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:39,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D903B4CBCC1749C789EA91097CCB0A85, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:39,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:39,915 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F4D8C8056A184D43958C958556BB3880, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:39,915 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:39,915 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F4D8C8056A184D43958C958556BB3880
2016-05-17 21:29:39,915 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:39,915 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F4D8C8056A184D43958C958556BB3880
2016-05-17 21:29:39,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:39,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D903B4CBCC1749C789EA91097CCB0A85
2016-05-17 21:29:39,917 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:39,919 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:39,919 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:39,919 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D903B4CBCC1749C789EA91097CCB0A85
2016-05-17 21:29:39,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D903B4CBCC1749C789EA91097CCB0A85
2016-05-17 21:29:39,923 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AE9F50BD7CB94B9794FFD82A805430B2, ordinal: 1
2016-05-17 21:29:39,923 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AE9F50BD7CB94B9794FFD82A805430B2, mem on start (mb), free: 115, total: 361, max: 672
2016-05-17 21:29:39,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:39,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:39,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:39,925 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:39,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F4D8C8056A184D43958C958556BB3880
2016-05-17 21:29:39,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:39,926 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:39,926 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:39,926 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:39,926 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:39,926 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:39,927 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:39,927 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F4D8C8056A184D43958C958556BB3880, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:39,927 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D903B4CBCC1749C789EA91097CCB0A85, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:39,927 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:39,927 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D903B4CBCC1749C789EA91097CCB0A85 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:39,927 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D903B4CBCC1749C789EA91097CCB0A85
2016-05-17 21:29:39,927 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:39,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:39,928 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:39,928 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:39,929 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D903B4CBCC1749C789EA91097CCB0A85 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:39,929 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:39,929 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:39,929 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:39,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D903B4CBCC1749C789EA91097CCB0A85
2016-05-17 21:29:39,932 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:39,932 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520579668_0001_1_01_000000_0
2016-05-17 21:29:39,936 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:39,937 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520579668_0001_1_00_000000_0_10159, runDuration: 35035, ]
2016-05-17 21:29:39,937 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 3AC38C0BCAFD4A68B444839B903B8657
2016-05-17 21:29:39,937 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_ALL"], id: F4D8C8056A184D43958C958556BB3880
2016-05-17 21:29:39,937 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_ALL"]
2016-05-17 21:29:39,938 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 3AC38C0BCAFD4A68B444839B903B8657, for 1 inputs
2016-05-17 21:29:39,938 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520579668_0001_1_00_000000_0_10159] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520579938, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:39,938 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D903B4CBCC1749C789EA91097CCB0A85
2016-05-17 21:29:39,938 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:39,939 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D903B4CBCC1749C789EA91097CCB0A85] #27 in 1ms
2016-05-17 21:29:39,939 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D903B4CBCC1749C789EA91097CCB0A85] #27, status:false, isInterrupted:false
2016-05-17 21:29:39,939 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:39,940 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:39,940 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:39,940 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:39,940 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:39,941 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:39,941 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AE9F50BD7CB94B9794FFD82A805430B2, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:39,941 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D903B4CBCC1749C789EA91097CCB0A85
2016-05-17 21:29:39,942 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:39,942 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:39,944 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@52bdd960; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@2e0a5810
2016-05-17 21:29:39,947 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:39,947 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:39,948 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520579668_0001_1_01_000000_0
2016-05-17 21:29:39,948 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520579668_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:39,949 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520579668_0001_1_01_000000_0
2016-05-17 21:29:39,949 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AE9F50BD7CB94B9794FFD82A805430B2, mem on close (mb), free: 112, total: 361, max: 672
2016-05-17 21:29:39,949 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D903B4CBCC1749C789EA91097CCB0A85
2016-05-17 21:29:39,950 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:39,950 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:39,951 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:39,951 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:39,951 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:39,951 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:39 Completed running task attempt: attempt_1463520579668_0001_1_01_000000_0
2016-05-17 21:29:39,952 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520579668_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:39,952 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:39,953 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AE9F50BD7CB94B9794FFD82A805430B2, taskAttemptId=attempt_1463520579668_0001_1_01_000000_0, startTime=1463520579911, finishTime=1463520579952, timeTaken=41, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=758, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520579941, Process_Duration=8, Process_End_Time=1463520579949, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:39,953 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520579668_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:39,953 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520579668_0001_1_01_000000_0
2016-05-17 21:29:39,953 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:39,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:39,953 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:TASK_FINISHED]: vertexName=AE9F50BD7CB94B9794FFD82A805430B2, taskId=task_1463520579668_0001_1_01_000000, startTime=1463520579911, finishTime=1463520579953, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1463520579668_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=758, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520579941, Process_Duration=8, Process_End_Time=1463520579949, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:39,954 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520579668_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:39,954 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2] : 1
2016-05-17 21:29:39,954 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2]
2016-05-17 21:29:39,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:39,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:VERTEX_FINISHED]: vertexName=AE9F50BD7CB94B9794FFD82A805430B2, vertexId=vertex_1463520579668_0001_1_01, initRequestedTime=1463520579831, initedTime=1463520579837, startRequestedTime=1463520579843, startedTime=1463520579843, finishTime=1463520579954, timeTaken=111, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=758, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520579941, Process_Duration=8, Process_End_Time=1463520579949, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520579911, firstTasksToStart=[ task_1463520579668_0001_1_01_000000 ], lastTaskFinishTime=1463520579952, lastTasksToFinish=[ task_1463520579668_0001_1_01_000000 ], minTaskDuration=41, maxTaskDuration=41, avgTaskDuration=41.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520579668_0001_1_01_000000 ], longestDurationTasks=[ task_1463520579668_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:39,954 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:39,955 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579668_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:39,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520579668_0001_00_000002, stoppedTime=1463520579955, exitStatus=0
2016-05-17 21:29:39,955 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579668_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:39,955 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:39,955 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520579668_0001_1
2016-05-17 21:29:39,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:39,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520579668_0001_1
2016-05-17 21:29:39,955 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: F4D8C8056A184D43958C958556BB3880 for vertex: vertex_1463520579668_0001_1_01 [AE9F50BD7CB94B9794FFD82A805430B2]
2016-05-17 21:29:39,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=758, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=378535936, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520579941, Process_Duration=8, Process_End_Time=1463520579949, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:39,958 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:39,958 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:39,958 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520579668_0001_00_000002
2016-05-17 21:29:39,958 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520579668_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:39,958 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520579668_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:39,959 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520579668_0001_00_000002, Exiting
2016-05-17 21:29:39,959 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520579668_0001_00_000002
2016-05-17 21:29:39,959 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520579668_0001_00_000002
2016-05-17 21:29:39,959 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520579668_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:39,959 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520579668_0001_00_000002 completed successfully
2016-05-17 21:29:39,961 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520579668_0001_1_00 [D903B4CBCC1749C789EA91097CCB0A85]
2016-05-17 21:29:39,963 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520579668_0001_1, queueSize=0
2016-05-17 21:29:39,964 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:39,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520579668_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520579668_0001_1, startTime=1463520579829, finishTime=1463520579961, timeTaken=132, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=971, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=757071872, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041159831, Process_Duration=10, Process_End_Time=2927041159841, Read_Duration=0, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:39,964 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520579668_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:39,965 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520579668_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:39,965 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520579668_0001_00_000002
2016-05-17 21:29:39,965 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520579668_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:39 Completed Dag: dag_1463520579668_0001_1
2016-05-17 21:29:39,965 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520579668_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:39,965 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520579668_0001_1_post
2016-05-17 21:29:39,965 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:39,965 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:39,970 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/0_UNKNOWN_ALL, applicationId=application_1463520579668_0001
2016-05-17 21:29:39,971 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:39,972 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:39,972 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:39,972 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:39,972 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:39,972 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:39,972 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:39,973 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:39,973 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:39,973 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:39,973 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:39,973 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:39,973 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:39,978 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/B68253108FB448778DE78A6B0F1AD518/CCE0AB81C6044CA5A76D5202408BD26C/.tez/application_1463520579668_0001
2016-05-17 21:29:39,978 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_ALL/_temporary
2016-05-17 21:29:39,978 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.327
2016-05-17 21:29:40,048 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_ALL
2016-05-17 21:29:40,067 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:40,067 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:40,097 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 21:29:40,106 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 21:29:40,107 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:40,107 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:40,107 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:40,124 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:40,129 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_REPLACE
2016-05-17 21:29:40,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:40,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:40,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_REPLACE"]
2016-05-17 21:29:40,169 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:40,170 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:40,170 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:40,171 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...latform/0_UNKNOWN_REPLACE
2016-05-17 21:29:40,171 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:40,182 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:40,186 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:40,187 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:40,187 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:40,196 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FBA653CFC12B49898C65874EA17F6095/BE1AF0B6A7A042E9872F8B94D3E97C98/.tez/application_1463520580186_0001 doesn't exist and is created
2016-05-17 21:29:40,210 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:40,210 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:40,211 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FBA653CFC12B49898C65874EA17F6095/BE1AF0B6A7A042E9872F8B94D3E97C98/.tez/application_1463520580186_0001
2016-05-17 21:29:40,216 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520580186_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:40,231 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:40,232 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:40,234 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:40,234 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:40,234 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FBA653CFC12B49898C65874EA17F6095/BE1AF0B6A7A042E9872F8B94D3E97C98/.tez/application_1463520580186_0001/localmode-log-dir
2016-05-17 21:29:40,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FBA653CFC12B49898C65874EA17F6095/BE1AF0B6A7A042E9872F8B94D3E97C98/.tez/application_1463520580186_0001/localmode-log-dir/history.txt.appattempt_1463520580186_0001_000000, maxErrors=10
2016-05-17 21:29:40,235 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:40,235 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520580186_0001_000000, appSubmitTime=1463520580215, launchTime=1463520580216
2016-05-17 21:29:40,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:40,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:40,242 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:47765
2016-05-17 21:29:40,243 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:40,243 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:40,244 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:40,246 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520580186_0001_000000, startTime=1463520580246
2016-05-17 21:29:40,246 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:40,246 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:40,312 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:40,312 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:40,312 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_REPLACE, applicationId=application_1463520580186_0001, dagName=[FBA653CFC12B49898C65874EA17F6095/BE1AF0B6A7A042E9872F8B94D3E97C98] every/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-05-17 21:29:40,327 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [FBA653CFC12B49898C65874EA17F6095/BE1AF0B6A7A042E9872F8B94D3E97C98] every/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-05-17 21:29:40,328 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520580186_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FBA653CFC12B49898C65874EA17F6095/BE1AF0B6A7A042E9872F8B94D3E97C98/.tez/application_1463520580186_0001/localmode-log-dir/dag_1463520580186_0001_1.dot
2016-05-17 21:29:40,329 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FBA653CFC12B49898C65874EA17F6095/BE1AF0B6A7A042E9872F8B94D3E97C98/.tez/application_1463520580186_0001/localmode-log-dir/dag_1463520580186_0001_1-tez-dag.pb.txt
2016-05-17 21:29:40,335 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520580186_0001_1
2016-05-17 21:29:40,335 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:40,335 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [FBA653CFC12B49898C65874EA17F6095/BE1AF0B6A7A042E9872F8B94D3E97C98] every/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-05-17 21:29:40 Running Dag: dag_1463520580186_0001_1
2016-05-17 21:29:40,345 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520580186_0001_1, submitTime=1463520580327
2016-05-17 21:29:40,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:40,346 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E16212C00BE24E1E84883A3FC34B50AA
2016-05-17 21:29:40,346 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:40,347 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520580186_0001_1, initTime=1463520580346
2016-05-17 21:29:40,347 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520580186_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:40,347 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:40,347 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:40,347 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:DAG_STARTED]: dagID=dag_1463520580186_0001_1, startTime=1463520580347
2016-05-17 21:29:40,347 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:40,347 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520580186_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:40,347 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BE7C8BC7349144E0875EDB60EEAE76B4 : {60BF1FA8A45E42FCA38FA875C6C3EE18={InputName=60BF1FA8A45E42FCA38FA875C6C3EE18}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:40,347 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 60BF1FA8A45E42FCA38FA875C6C3EE18, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:40,347 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4]
2016-05-17 21:29:40,347 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4]
2016-05-17 21:29:40,348 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4]
2016-05-17 21:29:40,348 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4]. Starting root input initializers: 1
2016-05-17 21:29:40,348 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:40,348 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 60BF1FA8A45E42FCA38FA875C6C3EE18 on vertex vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4]
2016-05-17 21:29:40,348 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA]
2016-05-17 21:29:40,350 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:40,350 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_REPLACE, applicationId=application_1463520580186_0001, dagName=[FBA653CFC12B49898C65874EA17F6095/BE1AF0B6A7A042E9872F8B94D3E97C98] every/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-05-17 21:29:40,350 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 60BF1FA8A45E42FCA38FA875C6C3EE18 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:40,350 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:40,350 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:40,351 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:40,351 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA]
2016-05-17 21:29:40,351 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA]
2016-05-17 21:29:40,352 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA]
2016-05-17 21:29:40,352 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=BCBF09A9A1124F689F6CE7E8954583D6, vertexId=vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:40,352 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=BCBF09A9A1124F689F6CE7E8954583D6, vertexId=vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA]
2016-05-17 21:29:40,354 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=BCBF09A9A1124F689F6CE7E8954583D6, vertexName=E16212C00BE24E1E84883A3FC34B50AA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:40,354 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=BCBF09A9A1124F689F6CE7E8954583D6, vertexId=vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA]
2016-05-17 21:29:40,355 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:40,355 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E16212C00BE24E1E84883A3FC34B50AA, vertexId=vertex_1463520580186_0001_1_01, initRequestedTime=1463520580348, initedTime=1463520580355, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:40,355 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:40,355 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:40,356 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 60BF1FA8A45E42FCA38FA875C6C3EE18 on vertex vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4]
2016-05-17 21:29:40,356 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:40,356 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {60BF1FA8A45E42FCA38FA875C6C3EE18=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:40,356 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4] parallelism set to 1
2016-05-17 21:29:40,356 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:40,356 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520580186_0001_1_00, eventCount=1
2016-05-17 21:29:40,357 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BE7C8BC7349144E0875EDB60EEAE76B4, vertexId=vertex_1463520580186_0001_1_00, initRequestedTime=1463520580347, initedTime=1463520580357, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:40,357 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:40,357 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4] with distanceFromRoot: 0
2016-05-17 21:29:40,357 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:40,357 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BE7C8BC7349144E0875EDB60EEAE76B4
2016-05-17 21:29:40,357 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4]
2016-05-17 21:29:40,358 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:40,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520580186_0001_1_00, startRequestedTime=1463520580357, startedTime=1463520580357
2016-05-17 21:29:40,358 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:40,358 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:40,358 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:40,358 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:40,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:TASK_STARTED]: vertexName=BE7C8BC7349144E0875EDB60EEAE76B4, taskId=task_1463520580186_0001_1_00_000000, scheduledTime=1463520580358, launchTime=1463520580358
2016-05-17 21:29:40,359 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520580186_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:40,359 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520580186_0001_1_00 for vertex: vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA] numStartedSources: 1 numSources: 1
2016-05-17 21:29:40,359 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA] with distanceFromRoot: 1
2016-05-17 21:29:40,359 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4]
2016-05-17 21:29:40,360 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BE7C8BC7349144E0875EDB60EEAE76B4 in vertex: E16212C00BE24E1E84883A3FC34B50AA
2016-05-17 21:29:40,360 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E16212C00BE24E1E84883A3FC34B50AA with 1 source tasks and 1 pending tasks
2016-05-17 21:29:40,360 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:40,360 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520580186_0001_1_01, startRequestedTime=1463520580359, startedTime=1463520580359
2016-05-17 21:29:40,361 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:40,361 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FBA653CFC12B49898C65874EA17F6095/BE1AF0B6A7A042E9872F8B94D3E97C98] every/(1/1) ...latform/0_UNKNOWN_REPLACE, VertexName: BE7C8BC7349144E0875EDB60EEAE76B4, VertexParallelism: 1, TaskAttemptID:attempt_1463520580186_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=60BF1FA8A45E42FCA38FA875C6C3EE18, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E16212C00BE24E1E84883A3FC34B50AA, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:40,361 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520580186_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:40,361 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:40,362 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:40,362 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:40,362 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:40,362 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520580186_0001_00_000001
2016-05-17 21:29:40,362 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580186_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:40,364 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520580186_0001_00_000001
2016-05-17 21:29:40,364 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:40,365 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580186_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:40,365 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:40,365 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520580186_0001_00_000001, launchTime=1463520580365
2016-05-17 21:29:40,366 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520580186_0001_00_000001
2016-05-17 21:29:40,366 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520580186_0001_1_00_000000_0] to container: [container_1463520580186_0001_00_000001]
2016-05-17 21:29:40,366 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580186_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:40,367 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520580186_0001_1_00_000000_0] started. Is using containerId: [container_1463520580186_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:40,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BE7C8BC7349144E0875EDB60EEAE76B4, taskAttemptId=attempt_1463520580186_0001_1_00_000000_0, startTime=1463520580366, containerId=container_1463520580186_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520580186_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:40,367 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520580186_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:40,367 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520580186_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:40,367 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:40,367 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520580186_0001_00_000001 given task: attempt_1463520580186_0001_1_00_000000_0
2016-05-17 21:29:40,367 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520580186_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520580186_0001_1_00_000000_0
2016-05-17 21:29:40 Starting to run new task attempt: attempt_1463520580186_0001_1_00_000000_0
2016-05-17 21:29:40,368 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520580186_0001_1_00_000000_0
2016-05-17 21:29:40,368 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:40,368 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:40,368 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:40,368 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:40,368 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:40,368 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:40,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FBA653CFC12B49898C65874EA17F6095/BE1AF0B6A7A042E9872F8B94D3E97C98] every/(1/1) ...latform/0_UNKNOWN_REPLACE, VertexName: BE7C8BC7349144E0875EDB60EEAE76B4, VertexParallelism: 1, TaskAttemptID:attempt_1463520580186_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=60BF1FA8A45E42FCA38FA875C6C3EE18, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E16212C00BE24E1E84883A3FC34B50AA, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:40,369 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:40,369 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520580186_0001, with dag id: dag_1463520580186_0001_1
2016-05-17 21:29:40,369 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520580186_0001_1_00_000000_0
2016-05-17 21:29:40,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:40,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:40,370 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520580186_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:40,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=60BF1FA8A45E42FCA38FA875C6C3EE18, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:40,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:40,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 60BF1FA8A45E42FCA38FA875C6C3EE18
2016-05-17 21:29:40,371 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 60BF1FA8A45E42FCA38FA875C6C3EE18
2016-05-17 21:29:40,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:40,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E16212C00BE24E1E84883A3FC34B50AA, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:40,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:40,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E16212C00BE24E1E84883A3FC34B50AA
2016-05-17 21:29:40,373 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:40,373 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E16212C00BE24E1E84883A3FC34B50AA
2016-05-17 21:29:40,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E16212C00BE24E1E84883A3FC34B50AA
2016-05-17 21:29:40,374 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:40,375 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:40,375 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 60BF1FA8A45E42FCA38FA875C6C3EE18
2016-05-17 21:29:40,375 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 60BF1FA8A45E42FCA38FA875C6C3EE18
2016-05-17 21:29:40,380 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BE7C8BC7349144E0875EDB60EEAE76B4, ordinal: 0
2016-05-17 21:29:40,380 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE7C8BC7349144E0875EDB60EEAE76B4, mem on start (mb), free: 239, total: 362, max: 672
2016-05-17 21:29:40,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:40,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:40,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:40,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:40,381 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:40,381 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:40,381 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:40,382 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:40,382 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:40,382 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:40,382 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 60BF1FA8A45E42FCA38FA875C6C3EE18, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:40,382 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E16212C00BE24E1E84883A3FC34B50AA, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:40,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:40,383 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 60BF1FA8A45E42FCA38FA875C6C3EE18 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:40,383 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:40,383 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:40,383 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 60BF1FA8A45E42FCA38FA875C6C3EE18
2016-05-17 21:29:40,383 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:40,383 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 60BF1FA8A45E42FCA38FA875C6C3EE18
2016-05-17 21:29:40,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:40,384 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520580186_0001_1_00_000000_0
2016-05-17 21:29:40,384 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:40,384 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2b0fd239
2016-05-17 21:29:40,385 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2b0fd239
2016-05-17 21:29:40,385 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:40,385 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:40,388 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:40,389 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 60BF1FA8A45E42FCA38FA875C6C3EE18
2016-05-17 21:29:40,389 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: F676E50E38E548458FD243291514B2DC
2016-05-17 21:29:40,389 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] F676E50E38E548458FD243291514B2DC
2016-05-17 21:29:40,390 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:40,393 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:40,393 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@1d801cf2; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@45ec62ff; comparator=cascading.tuple.hadoop.util.TupleComparator@22577736; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:40,393 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:40,394 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:40,394 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:40,407 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:40,407 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:40,408 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:40,408 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:40,408 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:40,408 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:40,409 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:40,409 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BE7C8BC7349144E0875EDB60EEAE76B4, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:40,412 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520580186_0001_1_00_000000_0
2016-05-17 21:29:40,412 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BE7C8BC7349144E0875EDB60EEAE76B4, mem on close (mb), free: 137, total: 362, max: 672
2016-05-17 21:29:40,412 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:40,412 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:40,412 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:40,412 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:40,416 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:40 Completed running task attempt: attempt_1463520580186_0001_1_00_000000_0
2016-05-17 21:29:40,420 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520580186_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:40,420 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520580186_0001_1_00, eventCount=1
2016-05-17 21:29:40,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520580409, Process_Duration=3, Process_End_Time=1463520580412, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:40,420 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:40,421 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:40,421 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520580186_0001_00_000001
2016-05-17 21:29:40,421 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:40,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BE7C8BC7349144E0875EDB60EEAE76B4, taskAttemptId=attempt_1463520580186_0001_1_00_000000_0, startTime=1463520580366, finishTime=1463520580420, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520580409, Process_Duration=3, Process_End_Time=1463520580412, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:40,421 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520580186_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:40,422 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520580186_0001_1_00_000000_0
2016-05-17 21:29:40,422 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:40,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:TASK_FINISHED]: vertexName=BE7C8BC7349144E0875EDB60EEAE76B4, taskId=task_1463520580186_0001_1_00_000000, startTime=1463520580366, finishTime=1463520580422, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1463520580186_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520580409, Process_Duration=3, Process_End_Time=1463520580412, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:40,422 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520580186_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:40,422 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4] : 1
2016-05-17 21:29:40,422 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4]
2016-05-17 21:29:40,423 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:40,423 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:VERTEX_FINISHED]: vertexName=BE7C8BC7349144E0875EDB60EEAE76B4, vertexId=vertex_1463520580186_0001_1_00, initRequestedTime=1463520580347, initedTime=1463520580357, startRequestedTime=1463520580357, startedTime=1463520580357, finishTime=1463520580422, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520580409, Process_Duration=3, Process_End_Time=1463520580412, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520580366, firstTasksToStart=[ task_1463520580186_0001_1_00_000000 ], lastTaskFinishTime=1463520580420, lastTasksToFinish=[ task_1463520580186_0001_1_00_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520580186_0001_1_00_000000 ], longestDurationTasks=[ task_1463520580186_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:40,423 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:40,423 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA] attempt: attempt_1463520580186_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:40,424 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E16212C00BE24E1E84883A3FC34B50AA
2016-05-17 21:29:40,424 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA]
2016-05-17 21:29:40,424 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:40,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:TASK_STARTED]: vertexName=E16212C00BE24E1E84883A3FC34B50AA, taskId=task_1463520580186_0001_1_01_000000, scheduledTime=1463520580424, launchTime=1463520580424
2016-05-17 21:29:40,424 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520580186_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:40,424 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:40,424 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:40,424 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA]
2016-05-17 21:29:40,425 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580186_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:40,425 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520580186_0001_00_000001, stoppedTime=1463520580425, exitStatus=0
2016-05-17 21:29:40,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:40,426 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580186_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:40,426 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FBA653CFC12B49898C65874EA17F6095/BE1AF0B6A7A042E9872F8B94D3E97C98] every/(1/1) ...latform/0_UNKNOWN_REPLACE, VertexName: E16212C00BE24E1E84883A3FC34B50AA, VertexParallelism: 1, TaskAttemptID:attempt_1463520580186_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BE7C8BC7349144E0875EDB60EEAE76B4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=BCBF09A9A1124F689F6CE7E8954583D6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:40,426 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520580186_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:40,426 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520580186_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:40,426 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:40,427 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580186_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:40,427 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520580186_0001_00_000002
2016-05-17 21:29:40,427 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580186_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:40,428 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520580186_0001_00_000002
2016-05-17 21:29:40,429 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:40,429 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580186_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:40,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:40,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520580186_0001_00_000002, launchTime=1463520580429
2016-05-17 21:29:40,430 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520580186_0001_00_000002
2016-05-17 21:29:40,431 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520580186_0001_1_01_000000_0] to container: [container_1463520580186_0001_00_000002]
2016-05-17 21:29:40,431 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580186_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:40,431 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520580186_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:40,431 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520580186_0001_1_01_000000_0] started. Is using containerId: [container_1463520580186_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:40,431 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520580186_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:40,432 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520580186_0001_00_000001, Exiting
2016-05-17 21:29:40,432 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:40,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E16212C00BE24E1E84883A3FC34B50AA, taskAttemptId=attempt_1463520580186_0001_1_01_000000_0, startTime=1463520580431, containerId=container_1463520580186_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520580186_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:40,432 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520580186_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:40,432 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520580186_0001_00_000001
2016-05-17 21:29:40,432 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520580186_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:40,432 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520580186_0001_00_000002 given task: attempt_1463520580186_0001_1_01_000000_0
2016-05-17 21:29:40,432 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520580186_0001_00_000001
2016-05-17 21:29:40,433 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520580186_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520580186_0001_1_01_000000_0
2016-05-17 21:29:40 Starting to run new task attempt: attempt_1463520580186_0001_1_01_000000_0
2016-05-17 21:29:40,433 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520580186_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:40,433 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520580186_0001_1_01_000000_0
2016-05-17 21:29:40,434 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:40,434 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:40,434 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:40,434 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:40,435 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:40,435 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:40,435 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FBA653CFC12B49898C65874EA17F6095/BE1AF0B6A7A042E9872F8B94D3E97C98] every/(1/1) ...latform/0_UNKNOWN_REPLACE, VertexName: E16212C00BE24E1E84883A3FC34B50AA, VertexParallelism: 1, TaskAttemptID:attempt_1463520580186_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BE7C8BC7349144E0875EDB60EEAE76B4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=BCBF09A9A1124F689F6CE7E8954583D6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:40,435 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:40,434 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520580186_0001_00_000001 completed successfully
2016-05-17 21:29:40,435 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580186_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:40,435 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520580186_0001_1_01_000000_0
2016-05-17 21:29:40,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:40,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:40,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BE7C8BC7349144E0875EDB60EEAE76B4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:40,437 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520580186_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:40,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:40,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:40,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BE7C8BC7349144E0875EDB60EEAE76B4
2016-05-17 21:29:40,439 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:40,439 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:40,439 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:40,440 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BE7C8BC7349144E0875EDB60EEAE76B4
2016-05-17 21:29:40,441 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BE7C8BC7349144E0875EDB60EEAE76B4
2016-05-17 21:29:40,444 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BCBF09A9A1124F689F6CE7E8954583D6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:40,444 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:40,444 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BCBF09A9A1124F689F6CE7E8954583D6
2016-05-17 21:29:40,444 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:40,445 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: BCBF09A9A1124F689F6CE7E8954583D6
2016-05-17 21:29:40,445 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E16212C00BE24E1E84883A3FC34B50AA, ordinal: 1
2016-05-17 21:29:40,445 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E16212C00BE24E1E84883A3FC34B50AA, mem on start (mb), free: 132, total: 362, max: 672
2016-05-17 21:29:40,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:40,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:40,445 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:40,456 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:40,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BCBF09A9A1124F689F6CE7E8954583D6
2016-05-17 21:29:40,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:40,456 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:40,456 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:40,456 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:40,456 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:40,456 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:40,457 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:40,457 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BE7C8BC7349144E0875EDB60EEAE76B4, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:40,457 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BCBF09A9A1124F689F6CE7E8954583D6, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:40,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:40,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BE7C8BC7349144E0875EDB60EEAE76B4 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:40,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:40,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:40,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BE7C8BC7349144E0875EDB60EEAE76B4
2016-05-17 21:29:40,458 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:40,458 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:40,458 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BE7C8BC7349144E0875EDB60EEAE76B4 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:40,459 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:40,459 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:40,459 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:40,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BE7C8BC7349144E0875EDB60EEAE76B4
2016-05-17 21:29:40,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:40,462 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520580186_0001_1_01_000000_0
2016-05-17 21:29:40,463 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520580186_0001_1_00_000000_0_10165, runDuration: 46462, ]
2016-05-17 21:29:40,465 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:40,465 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520580186_0001_1_00_000000_0_10165] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520580465, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:40,465 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BE7C8BC7349144E0875EDB60EEAE76B4
2016-05-17 21:29:40,465 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: F676E50E38E548458FD243291514B2DC
2016-05-17 21:29:40,466 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_REPLACE"], id: BCBF09A9A1124F689F6CE7E8954583D6
2016-05-17 21:29:40,466 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:40,466 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_REPLACE"]
2016-05-17 21:29:40,466 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [BE7C8BC7349144E0875EDB60EEAE76B4] #28 in 3ms
2016-05-17 21:29:40,466 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] F676E50E38E548458FD243291514B2DC, for 1 inputs
2016-05-17 21:29:40,466 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BE7C8BC7349144E0875EDB60EEAE76B4] #28, status:false, isInterrupted:false
2016-05-17 21:29:40,467 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:40,468 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:40,468 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:40,468 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:40,468 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:40,468 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:40,469 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BE7C8BC7349144E0875EDB60EEAE76B4
2016-05-17 21:29:40,469 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:40,469 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E16212C00BE24E1E84883A3FC34B50AA, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:40,470 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:40,472 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@750343fe; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@296514b6
2016-05-17 21:29:40,473 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:40,475 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:40,477 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:40,477 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:40,477 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520580186_0001_1_01_000000_0
2016-05-17 21:29:40,478 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520580186_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:40,478 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520580186_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:40,480 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E16212C00BE24E1E84883A3FC34B50AA, taskAttemptId=attempt_1463520580186_0001_1_01_000000_0, startTime=1463520580431, finishTime=1463520580480, timeTaken=49, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520580469, Process_Duration=9, Process_End_Time=1463520580478, Read_Duration=0, Tuples_Read=1
2016-05-17 21:29:40,480 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520580186_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:40,480 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520580186_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:40,480 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:TASK_FINISHED]: vertexName=E16212C00BE24E1E84883A3FC34B50AA, taskId=task_1463520580186_0001_1_01_000000, startTime=1463520580431, finishTime=1463520580480, timeTaken=49, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:40,481 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520580186_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:40,481 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA] : 1
2016-05-17 21:29:40,481 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA] because task failed: task_1463520580186_0001_1_01_000000
2016-05-17 21:29:40,481 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:40,481 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:40,481 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA]
2016-05-17 21:29:40,481 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:40,481 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:40,481 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=BCBF09A9A1124F689F6CE7E8954583D6, vertexId=vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA]
2016-05-17 21:29:40,482 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:40,483 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:40,483 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:VERTEX_FINISHED]: vertexName=E16212C00BE24E1E84883A3FC34B50AA, vertexId=vertex_1463520580186_0001_1_01, initRequestedTime=1463520580348, initedTime=1463520580355, startRequestedTime=1463520580359, startedTime=1463520580359, finishTime=1463520580483, timeTaken=124, status=FAILED, diagnostics=Task failed, taskId=task_1463520580186_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520580431, firstTasksToStart=[ task_1463520580186_0001_1_01_000000 ], lastTaskFinishTime=1463520580480, lastTasksToFinish=[ task_1463520580186_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:40,483 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:40,484 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520580186_0001_00_000002, stoppedTime=1463520580484, exitStatus=0
2016-05-17 21:29:40,484 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580186_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:40,484 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520580186_0001_1_01_000000_0 failed: true container: container_1463520580186_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:40,484 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:40,484 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:40,484 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520580186_0001_1
2016-05-17 21:29:40,484 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: BCBF09A9A1124F689F6CE7E8954583D6 for vertex: vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA]
2016-05-17 21:29:40,484 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520580186_0001_1_00 [BE7C8BC7349144E0875EDB60EEAE76B4]
2016-05-17 21:29:40,485 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:40,485 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520580186_0001_1, queueSize=0
2016-05-17 21:29:40,485 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:40,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580186_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520580186_0001_1, startTime=1463520580347, finishTime=1463520580484, timeTaken=137, status=FAILED, diagnostics=Vertex failed, vertexName=E16212C00BE24E1E84883A3FC34B50AA, vertexId=vertex_1463520580186_0001_1_01, diagnostics=[Task failed, taskId=task_1463520580186_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520580186_0001_1_01 [E16212C00BE24E1E84883A3FC34B50AA] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520580409, Process_Duration=3, Process_End_Time=1463520580412, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:40,486 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520580186_0001_1 finished with state: FAILED
2016-05-17 21:29:40,486 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520580186_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:40,486 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520580186_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:40,486 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520580469, Process_Duration=9, Process_End_Time=1463520580478, Read_Duration=0, Tuples_Read=1]]
2016-05-17 21:29:40 Completed Dag: dag_1463520580186_0001_1
2016-05-17 21:29:40,487 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520580186_0001_1, dagState=FAILED
2016-05-17 21:29:40,487 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520580186_0001_1_post
2016-05-17 21:29:40,487 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:40,487 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:40,487 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E16212C00BE24E1E84883A3FC34B50AA, mem on close (mb), free: 127, total: 362, max: 672
2016-05-17 21:29:40,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=E16212C00BE24E1E84883A3FC34B50AA, index=1
2016-05-17 21:29:40,487 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580186_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:40,488 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BE7C8BC7349144E0875EDB60EEAE76B4
2016-05-17 21:29:40,488 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:40,489 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:40,489 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:40,489 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:40,489 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:40,489 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:40,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=E16212C00BE24E1E84883A3FC34B50AA, sourceVertex=BE7C8BC7349144E0875EDB60EEAE76B4
2016-05-17 21:29:40,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=E16212C00BE24E1E84883A3FC34B50AA, sourceVertex=BCBF09A9A1124F689F6CE7E8954583D6
2016-05-17 21:29:40,489 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520580186_0001_00_000002
2016-05-17 21:29:40,489 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520580186_0001_00_000002
2016-05-17 21:29:40,490 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520580186_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:40,490 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520580186_0001_00_000002 completed but with errors
2016-05-17 21:29:40,490 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520580186_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:40,490 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580186_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:40,491 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520580186_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:40,497 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_UNKNOWN_REPLACE, applicationId=application_1463520580186_0001
2016-05-17 21:29:40,499 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:40,499 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:40,499 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:40,499 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:40,499 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:40,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:40,500 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:40,500 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:40,500 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:40,500 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:40,500 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:40,501 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:40,501 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:40,505 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/FBA653CFC12B49898C65874EA17F6095/BE1AF0B6A7A042E9872F8B94D3E97C98/.tez/application_1463520580186_0001
2016-05-17 21:29:40,505 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:40,506 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...latform/0_UNKNOWN_REPLACE
2016-05-17 21:29:40,506 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:40,506 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_REPLACE/_temporary
2016-05-17 21:29:40,507 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.337
2016-05-17 21:29:40,507 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_REPLACE
2016-05-17 21:29:40,527 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:40,527 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:40,555 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 21:29:40,562 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-05-17 21:29:40,562 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:40,563 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:40,563 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:40,575 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:40,579 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_SWAP
2016-05-17 21:29:40,635 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:40,635 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:40,636 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_SWAP"]
2016-05-17 21:29:40,636 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:40,636 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:40,637 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:40,637 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-05-17 21:29:40,637 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:40,648 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:40,652 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:40,652 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:40,652 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:40,660 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2073482192C74A4CA4C23E76685B6C7E/DC68228C10C94D77AAF1BF2476147AED/.tez/application_1463520580652_0001 doesn't exist and is created
2016-05-17 21:29:40,675 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:40,675 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:40,676 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2073482192C74A4CA4C23E76685B6C7E/DC68228C10C94D77AAF1BF2476147AED/.tez/application_1463520580652_0001
2016-05-17 21:29:40,678 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520580652_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:40,687 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:40,688 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:40,689 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:40,689 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:40,689 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2073482192C74A4CA4C23E76685B6C7E/DC68228C10C94D77AAF1BF2476147AED/.tez/application_1463520580652_0001/localmode-log-dir
2016-05-17 21:29:40,689 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2073482192C74A4CA4C23E76685B6C7E/DC68228C10C94D77AAF1BF2476147AED/.tez/application_1463520580652_0001/localmode-log-dir/history.txt.appattempt_1463520580652_0001_000000, maxErrors=10
2016-05-17 21:29:40,689 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:40,690 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520580652_0001_000000, appSubmitTime=1463520580677, launchTime=1463520580677
2016-05-17 21:29:40,692 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:40,694 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:40,695 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:40,695 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:40,699 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:42322
2016-05-17 21:29:40,700 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:40,701 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520580652_0001_000000, startTime=1463520580701
2016-05-17 21:29:40,701 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:40,702 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:40,775 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:40,776 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:40,776 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_UNKNOWN_SWAP, applicationId=application_1463520580652_0001, dagName=[2073482192C74A4CA4C23E76685B6C7E/DC68228C10C94D77AAF1BF2476147AED] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-05-17 21:29:40,789 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2073482192C74A4CA4C23E76685B6C7E/DC68228C10C94D77AAF1BF2476147AED] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-05-17 21:29:40,790 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520580652_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2073482192C74A4CA4C23E76685B6C7E/DC68228C10C94D77AAF1BF2476147AED/.tez/application_1463520580652_0001/localmode-log-dir/dag_1463520580652_0001_1.dot
2016-05-17 21:29:40,791 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2073482192C74A4CA4C23E76685B6C7E/DC68228C10C94D77AAF1BF2476147AED/.tez/application_1463520580652_0001/localmode-log-dir/dag_1463520580652_0001_1-tez-dag.pb.txt
2016-05-17 21:29:40,797 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520580652_0001_1
2016-05-17 21:29:40,797 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:40,797 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2073482192C74A4CA4C23E76685B6C7E/DC68228C10C94D77AAF1BF2476147AED] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-05-17 21:29:40 Running Dag: dag_1463520580652_0001_1
2016-05-17 21:29:40,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520580652_0001_1, submitTime=1463520580789
2016-05-17 21:29:40,808 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:40,808 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D9521D5A4D9C4F4F83D060C1857F15A7
2016-05-17 21:29:40,809 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:40,809 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520580652_0001_1, initTime=1463520580808
2016-05-17 21:29:40,809 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:40,809 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520580652_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:40,809 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:DAG_STARTED]: dagID=dag_1463520580652_0001_1, startTime=1463520580809
2016-05-17 21:29:40,810 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:40,810 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520580652_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:40,810 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A8805CAD939C4BF3B617506BB3317B2D : {B2B984C97D3F4A3285F38C1A37CFDFF0={InputName=B2B984C97D3F4A3285F38C1A37CFDFF0}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:40,810 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B2B984C97D3F4A3285F38C1A37CFDFF0, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:40,810 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D]
2016-05-17 21:29:40,810 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D]
2016-05-17 21:29:40,810 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D]
2016-05-17 21:29:40,810 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D]. Starting root input initializers: 1
2016-05-17 21:29:40,810 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:40,811 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:40,811 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_UNKNOWN_SWAP, applicationId=application_1463520580652_0001, dagName=[2073482192C74A4CA4C23E76685B6C7E/DC68228C10C94D77AAF1BF2476147AED] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-05-17 21:29:40,811 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B2B984C97D3F4A3285F38C1A37CFDFF0 on vertex vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D]
2016-05-17 21:29:40,811 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7]
2016-05-17 21:29:40,812 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:40,812 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B2B984C97D3F4A3285F38C1A37CFDFF0 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:40,813 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:40,813 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:40,814 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:40,814 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7]
2016-05-17 21:29:40,814 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7]
2016-05-17 21:29:40,814 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:40,814 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7]
2016-05-17 21:29:40,814 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=95D3D4773291459D9645C30547B29022, vertexId=vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:40,814 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=95D3D4773291459D9645C30547B29022, vertexId=vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7]
2016-05-17 21:29:40,814 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B2B984C97D3F4A3285F38C1A37CFDFF0 on vertex vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D]
2016-05-17 21:29:40,816 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=95D3D4773291459D9645C30547B29022, vertexName=D9521D5A4D9C4F4F83D060C1857F15A7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:40,816 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=95D3D4773291459D9645C30547B29022, vertexId=vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7]
2016-05-17 21:29:40,817 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:40,817 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D9521D5A4D9C4F4F83D060C1857F15A7, vertexId=vertex_1463520580652_0001_1_01, initRequestedTime=1463520580811, initedTime=1463520580817, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:40,817 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:40,817 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:40,817 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B2B984C97D3F4A3285F38C1A37CFDFF0=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:40,818 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D] parallelism set to 1
2016-05-17 21:29:40,818 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:40,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520580652_0001_1_00, eventCount=1
2016-05-17 21:29:40,818 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:40,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A8805CAD939C4BF3B617506BB3317B2D, vertexId=vertex_1463520580652_0001_1_00, initRequestedTime=1463520580810, initedTime=1463520580818, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:40,819 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D] with distanceFromRoot: 0
2016-05-17 21:29:40,819 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:40,819 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A8805CAD939C4BF3B617506BB3317B2D
2016-05-17 21:29:40,819 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D]
2016-05-17 21:29:40,819 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:40,820 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520580652_0001_1_00, startRequestedTime=1463520580819, startedTime=1463520580819
2016-05-17 21:29:40,820 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:40,820 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:40,820 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:TASK_STARTED]: vertexName=A8805CAD939C4BF3B617506BB3317B2D, taskId=task_1463520580652_0001_1_00_000000, scheduledTime=1463520580820, launchTime=1463520580820
2016-05-17 21:29:40,820 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520580652_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:40,821 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520580652_0001_1_00 for vertex: vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7] numStartedSources: 1 numSources: 1
2016-05-17 21:29:40,821 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7] with distanceFromRoot: 1
2016-05-17 21:29:40,821 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D]
2016-05-17 21:29:40,821 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A8805CAD939C4BF3B617506BB3317B2D in vertex: D9521D5A4D9C4F4F83D060C1857F15A7
2016-05-17 21:29:40,821 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D9521D5A4D9C4F4F83D060C1857F15A7 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:40,822 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:40,822 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520580652_0001_1_01, startRequestedTime=1463520580821, startedTime=1463520580821
2016-05-17 21:29:40,822 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:40,822 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2073482192C74A4CA4C23E76685B6C7E/DC68228C10C94D77AAF1BF2476147AED] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP, VertexName: A8805CAD939C4BF3B617506BB3317B2D, VertexParallelism: 1, TaskAttemptID:attempt_1463520580652_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B2B984C97D3F4A3285F38C1A37CFDFF0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D9521D5A4D9C4F4F83D060C1857F15A7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:40,822 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520580652_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:40,823 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:40,823 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:40,823 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:40,823 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:40,824 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520580652_0001_00_000001
2016-05-17 21:29:40,824 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580652_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:40,825 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520580652_0001_00_000001
2016-05-17 21:29:40,825 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:40,826 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580652_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:40,826 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:40,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520580652_0001_00_000001, launchTime=1463520580826
2016-05-17 21:29:40,827 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520580652_0001_00_000001
2016-05-17 21:29:40,827 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520580652_0001_1_00_000000_0] to container: [container_1463520580652_0001_00_000001]
2016-05-17 21:29:40,827 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580652_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:40,827 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520580652_0001_00_000001 given task: attempt_1463520580652_0001_1_00_000000_0
2016-05-17 21:29:40,828 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520580652_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520580652_0001_1_00_000000_0
2016-05-17 21:29:40,828 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520580652_0001_1_00_000000_0] started. Is using containerId: [container_1463520580652_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:40,828 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A8805CAD939C4BF3B617506BB3317B2D, taskAttemptId=attempt_1463520580652_0001_1_00_000000_0, startTime=1463520580827, containerId=container_1463520580652_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520580652_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:40 Starting to run new task attempt: attempt_1463520580652_0001_1_00_000000_0
2016-05-17 21:29:40,828 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520580652_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:40,828 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520580652_0001_1_00_000000_0
2016-05-17 21:29:40,828 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520580652_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:40,828 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:40,828 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:40,828 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:40,828 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:40,829 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:40,829 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:40,829 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:40,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2073482192C74A4CA4C23E76685B6C7E/DC68228C10C94D77AAF1BF2476147AED] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP, VertexName: A8805CAD939C4BF3B617506BB3317B2D, VertexParallelism: 1, TaskAttemptID:attempt_1463520580652_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B2B984C97D3F4A3285F38C1A37CFDFF0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D9521D5A4D9C4F4F83D060C1857F15A7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:40,829 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:40,829 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520580652_0001_1_00_000000_0
2016-05-17 21:29:40,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:40,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:40,830 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520580652_0001, with dag id: dag_1463520580652_0001_1
2016-05-17 21:29:40,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B2B984C97D3F4A3285F38C1A37CFDFF0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:40,831 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520580652_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:40,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:40,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B2B984C97D3F4A3285F38C1A37CFDFF0
2016-05-17 21:29:40,832 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B2B984C97D3F4A3285F38C1A37CFDFF0
2016-05-17 21:29:40,833 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:40,833 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B2B984C97D3F4A3285F38C1A37CFDFF0
2016-05-17 21:29:40,833 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B2B984C97D3F4A3285F38C1A37CFDFF0
2016-05-17 21:29:40,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:40,835 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:40,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D9521D5A4D9C4F4F83D060C1857F15A7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:40,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:40,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D9521D5A4D9C4F4F83D060C1857F15A7
2016-05-17 21:29:40,838 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:40,838 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D9521D5A4D9C4F4F83D060C1857F15A7
2016-05-17 21:29:40,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D9521D5A4D9C4F4F83D060C1857F15A7
2016-05-17 21:29:40,840 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A8805CAD939C4BF3B617506BB3317B2D, ordinal: 0
2016-05-17 21:29:40,840 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A8805CAD939C4BF3B617506BB3317B2D, mem on start (mb), free: 266, total: 362, max: 672
2016-05-17 21:29:40,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:40,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:40,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:40,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:40,840 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:40,840 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:40,840 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:40,840 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:40,841 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:40,841 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:40,841 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B2B984C97D3F4A3285F38C1A37CFDFF0, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:40,841 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D9521D5A4D9C4F4F83D060C1857F15A7, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:40,841 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:40,841 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B2B984C97D3F4A3285F38C1A37CFDFF0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:40,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:40,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B2B984C97D3F4A3285F38C1A37CFDFF0
2016-05-17 21:29:40,842 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:40,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B2B984C97D3F4A3285F38C1A37CFDFF0
2016-05-17 21:29:40,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:40,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:40,842 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520580652_0001_1_00_000000_0
2016-05-17 21:29:40,842 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:40,843 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4a356a40
2016-05-17 21:29:40,844 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4a356a40
2016-05-17 21:29:40,844 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:40,844 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:40,845 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:40,846 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B2B984C97D3F4A3285F38C1A37CFDFF0
2016-05-17 21:29:40,846 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: D12ADB6A13F34D9CA33A71E88646F4A1
2016-05-17 21:29:40,846 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] D12ADB6A13F34D9CA33A71E88646F4A1
2016-05-17 21:29:40,847 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:40,848 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:40,849 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@47a56ef1; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@3e16b4f6; comparator=cascading.tuple.hadoop.util.TupleComparator@44c2a252; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:40,849 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:40,849 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:40,849 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:40,862 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:40,862 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:40,862 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:40,862 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:40,862 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:40,863 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:40,863 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:40,863 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A8805CAD939C4BF3B617506BB3317B2D, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:40,866 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520580652_0001_1_00_000000_0
2016-05-17 21:29:40,866 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A8805CAD939C4BF3B617506BB3317B2D, mem on close (mb), free: 163, total: 362, max: 672
2016-05-17 21:29:40,866 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:40,866 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:40,866 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:40,867 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:40,870 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:40 Completed running task attempt: attempt_1463520580652_0001_1_00_000000_0
2016-05-17 21:29:40,872 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520580652_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:40,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520580652_0001_1_00, eventCount=1
2016-05-17 21:29:40,873 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:40,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A8805CAD939C4BF3B617506BB3317B2D, taskAttemptId=attempt_1463520580652_0001_1_00_000000_0, startTime=1463520580827, finishTime=1463520580873, timeTaken=46, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520580863, Process_Duration=2, Process_End_Time=1463520580865, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:40,873 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520580652_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:40,874 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520580652_0001_1_00_000000_0
2016-05-17 21:29:40,874 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:40,874 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:40,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:TASK_FINISHED]: vertexName=A8805CAD939C4BF3B617506BB3317B2D, taskId=task_1463520580652_0001_1_00_000000, startTime=1463520580827, finishTime=1463520580874, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1463520580652_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520580863, Process_Duration=2, Process_End_Time=1463520580865, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:40,874 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520580652_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:40,874 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D] : 1
2016-05-17 21:29:40,874 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D]
2016-05-17 21:29:40,875 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520580863, Process_Duration=2, Process_End_Time=1463520580865, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:40,875 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:40,875 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:40,875 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:40,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:VERTEX_FINISHED]: vertexName=A8805CAD939C4BF3B617506BB3317B2D, vertexId=vertex_1463520580652_0001_1_00, initRequestedTime=1463520580810, initedTime=1463520580818, startRequestedTime=1463520580819, startedTime=1463520580819, finishTime=1463520580874, timeTaken=55, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520580863, Process_Duration=2, Process_End_Time=1463520580865, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520580827, firstTasksToStart=[ task_1463520580652_0001_1_00_000000 ], lastTaskFinishTime=1463520580873, lastTasksToFinish=[ task_1463520580652_0001_1_00_000000 ], minTaskDuration=46, maxTaskDuration=46, avgTaskDuration=46.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520580652_0001_1_00_000000 ], longestDurationTasks=[ task_1463520580652_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:40,875 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520580652_0001_00_000001
2016-05-17 21:29:40,875 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:40,875 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580652_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:40,876 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520580652_0001_00_000001, stoppedTime=1463520580876, exitStatus=0
2016-05-17 21:29:40,876 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580652_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:40,876 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7] attempt: attempt_1463520580652_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:40,876 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D9521D5A4D9C4F4F83D060C1857F15A7
2016-05-17 21:29:40,876 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7]
2016-05-17 21:29:40,876 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:40,876 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:40,877 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520580652_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:40,877 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:TASK_STARTED]: vertexName=D9521D5A4D9C4F4F83D060C1857F15A7, taskId=task_1463520580652_0001_1_01_000000, scheduledTime=1463520580877, launchTime=1463520580877
2016-05-17 21:29:40,877 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520580652_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:40,877 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580652_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:40,877 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:40,877 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7]
2016-05-17 21:29:40,877 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2073482192C74A4CA4C23E76685B6C7E/DC68228C10C94D77AAF1BF2476147AED] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP, VertexName: D9521D5A4D9C4F4F83D060C1857F15A7, VertexParallelism: 1, TaskAttemptID:attempt_1463520580652_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A8805CAD939C4BF3B617506BB3317B2D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=95D3D4773291459D9645C30547B29022, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:40,878 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520580652_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:40,878 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:40,878 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520580652_0001_00_000002
2016-05-17 21:29:40,878 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580652_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:40,880 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520580652_0001_00_000002
2016-05-17 21:29:40,880 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:40,881 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:40,881 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580652_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:40,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520580652_0001_00_000002, launchTime=1463520580881
2016-05-17 21:29:40,882 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520580652_0001_00_000002
2016-05-17 21:29:40,882 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520580652_0001_1_01_000000_0] to container: [container_1463520580652_0001_00_000002]
2016-05-17 21:29:40,882 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580652_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:40,882 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520580652_0001_00_000002 given task: attempt_1463520580652_0001_1_01_000000_0
2016-05-17 21:29:40,883 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520580652_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520580652_0001_1_01_000000_0
2016-05-17 21:29:40,883 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520580652_0001_1_01_000000_0] started. Is using containerId: [container_1463520580652_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:40,883 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:40,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D9521D5A4D9C4F4F83D060C1857F15A7, taskAttemptId=attempt_1463520580652_0001_1_01_000000_0, startTime=1463520580883, containerId=container_1463520580652_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520580652_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:40,883 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520580652_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:40,883 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520580652_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:40 Starting to run new task attempt: attempt_1463520580652_0001_1_01_000000_0
2016-05-17 21:29:40,884 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520580652_0001_1_01_000000_0
2016-05-17 21:29:40,884 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:40,884 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:40,884 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:40,884 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:40,884 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:40,885 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:40,885 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2073482192C74A4CA4C23E76685B6C7E/DC68228C10C94D77AAF1BF2476147AED] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP, VertexName: D9521D5A4D9C4F4F83D060C1857F15A7, VertexParallelism: 1, TaskAttemptID:attempt_1463520580652_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A8805CAD939C4BF3B617506BB3317B2D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=95D3D4773291459D9645C30547B29022, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:40,885 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:40,885 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520580652_0001_1_01_000000_0
2016-05-17 21:29:40,886 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:40,886 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:40,886 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520580652_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:40,886 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520580652_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:40,886 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520580652_0001_00_000001, Exiting
2016-05-17 21:29:40,886 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520580652_0001_00_000001
2016-05-17 21:29:40,886 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520580652_0001_00_000001
2016-05-17 21:29:40,886 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520580652_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:40,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:40,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A8805CAD939C4BF3B617506BB3317B2D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:40,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:40,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A8805CAD939C4BF3B617506BB3317B2D
2016-05-17 21:29:40,896 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:40,896 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:40,897 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A8805CAD939C4BF3B617506BB3317B2D
2016-05-17 21:29:40,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=95D3D4773291459D9645C30547B29022, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:40,896 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:40,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A8805CAD939C4BF3B617506BB3317B2D
2016-05-17 21:29:40,897 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520580652_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:40,898 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520580652_0001_00_000001 completed successfully
2016-05-17 21:29:40,898 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580652_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:40,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:40,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 95D3D4773291459D9645C30547B29022
2016-05-17 21:29:40,899 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:40,900 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 95D3D4773291459D9645C30547B29022
2016-05-17 21:29:40,902 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D9521D5A4D9C4F4F83D060C1857F15A7, ordinal: 1
2016-05-17 21:29:40,902 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D9521D5A4D9C4F4F83D060C1857F15A7, mem on start (mb), free: 158, total: 362, max: 672
2016-05-17 21:29:40,902 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:40,902 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:40,903 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:40,908 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:40,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 95D3D4773291459D9645C30547B29022
2016-05-17 21:29:40,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:40,908 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:40,908 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:40,908 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:40,908 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:40,908 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:40,909 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:40,909 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A8805CAD939C4BF3B617506BB3317B2D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:40,909 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 95D3D4773291459D9645C30547B29022, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:40,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:40,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A8805CAD939C4BF3B617506BB3317B2D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:40,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A8805CAD939C4BF3B617506BB3317B2D
2016-05-17 21:29:40,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:40,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:40,910 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:40,910 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:40,910 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A8805CAD939C4BF3B617506BB3317B2D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:40,910 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:40,910 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:40,911 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:40,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A8805CAD939C4BF3B617506BB3317B2D
2016-05-17 21:29:40,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:40,912 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520580652_0001_1_01_000000_0
2016-05-17 21:29:40,914 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520580652_0001_1_00_000000_0_10171, runDuration: 34945, ]
2016-05-17 21:29:40,916 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:40,916 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: D12ADB6A13F34D9CA33A71E88646F4A1
2016-05-17 21:29:40,916 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_SWAP"], id: 95D3D4773291459D9645C30547B29022
2016-05-17 21:29:40,917 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_SWAP"]
2016-05-17 21:29:40,917 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] D12ADB6A13F34D9CA33A71E88646F4A1, for 1 inputs
2016-05-17 21:29:40,916 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520580652_0001_1_00_000000_0_10171] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520580916, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:40,917 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A8805CAD939C4BF3B617506BB3317B2D
2016-05-17 21:29:40,918 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:40,918 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [A8805CAD939C4BF3B617506BB3317B2D] #29 in 3ms
2016-05-17 21:29:40,918 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A8805CAD939C4BF3B617506BB3317B2D] #29, status:false, isInterrupted:false
2016-05-17 21:29:40,919 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:40,920 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:40,920 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:40,920 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:40,920 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:40,920 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:40,920 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A8805CAD939C4BF3B617506BB3317B2D
2016-05-17 21:29:40,920 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:40,920 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D9521D5A4D9C4F4F83D060C1857F15A7, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:40,921 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:40,923 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@4456f87b; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@29d3764a
2016-05-17 21:29:40,926 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:40,926 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:40,926 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520580652_0001_1_01_000000_0
2016-05-17 21:29:40,926 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520580652_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:40,927 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520580652_0001_1_01_000000_0
2016-05-17 21:29:40,927 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D9521D5A4D9C4F4F83D060C1857F15A7, mem on close (mb), free: 154, total: 362, max: 672
2016-05-17 21:29:40,927 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A8805CAD939C4BF3B617506BB3317B2D
2016-05-17 21:29:40,927 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:40,928 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:40,928 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:40,929 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:40,929 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:40,929 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:40 Completed running task attempt: attempt_1463520580652_0001_1_01_000000_0
2016-05-17 21:29:40,929 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520580652_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:40,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520580920, Process_Duration=7, Process_End_Time=1463520580927, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:40,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D9521D5A4D9C4F4F83D060C1857F15A7, taskAttemptId=attempt_1463520580652_0001_1_01_000000_0, startTime=1463520580883, finishTime=1463520580930, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520580920, Process_Duration=7, Process_End_Time=1463520580927, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:40,930 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:40,930 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:40,930 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520580652_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:40,931 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:40,931 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520580652_0001_1_01_000000_0
2016-05-17 21:29:40,931 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520580652_0001_00_000002
2016-05-17 21:29:40,931 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:TASK_FINISHED]: vertexName=D9521D5A4D9C4F4F83D060C1857F15A7, taskId=task_1463520580652_0001_1_01_000000, startTime=1463520580883, finishTime=1463520580931, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1463520580652_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520580920, Process_Duration=7, Process_End_Time=1463520580927, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:40,931 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520580652_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:40,931 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7] : 1
2016-05-17 21:29:40,932 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7]
2016-05-17 21:29:40,932 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:VERTEX_FINISHED]: vertexName=D9521D5A4D9C4F4F83D060C1857F15A7, vertexId=vertex_1463520580652_0001_1_01, initRequestedTime=1463520580811, initedTime=1463520580817, startRequestedTime=1463520580821, startedTime=1463520580821, finishTime=1463520580932, timeTaken=111, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520580920, Process_Duration=7, Process_End_Time=1463520580927, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520580883, firstTasksToStart=[ task_1463520580652_0001_1_01_000000 ], lastTaskFinishTime=1463520580930, lastTasksToFinish=[ task_1463520580652_0001_1_01_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520580652_0001_1_01_000000 ], longestDurationTasks=[ task_1463520580652_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:40,932 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:40,931 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:40,932 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:40,933 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:40,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:40,933 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520580652_0001_1
2016-05-17 21:29:40,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520580652_0001_1
2016-05-17 21:29:40,933 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520580652_0001_1_00 [A8805CAD939C4BF3B617506BB3317B2D]
2016-05-17 21:29:40,933 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 95D3D4773291459D9645C30547B29022 for vertex: vertex_1463520580652_0001_1_01 [D9521D5A4D9C4F4F83D060C1857F15A7]
2016-05-17 21:29:40,940 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520580652_0001_1, queueSize=0
2016-05-17 21:29:40,940 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:40,940 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520580652_0001_1, startTime=1463520580809, finishTime=1463520580938, timeTaken=129, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=396, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=760217600, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041161783, Process_Duration=9, Process_End_Time=2927041161792, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:40,941 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520580652_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:40,941 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520580652_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:40,941 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580652_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:40,942 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520580652_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520580652_0001_00_000002, stoppedTime=1463520580941, exitStatus=0
2016-05-17 21:29:40,942 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580652_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:40,942 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:40 Completed Dag: dag_1463520580652_0001_1
2016-05-17 21:29:40,943 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520580652_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:40,943 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520580652_0001_1_post
2016-05-17 21:29:40,943 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:40,943 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:40,943 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520580652_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:40,944 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520580652_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:40,947 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_UNKNOWN_SWAP, applicationId=application_1463520580652_0001
2016-05-17 21:29:40,949 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:40,949 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:40,949 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:40,949 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:40,950 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:40,950 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:40,951 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:40,951 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:40,952 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:40,952 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:40,952 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:40,952 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:40,952 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:40,953 INFO task.TezChild (TezChild.java:run(198)) - Interrupted while waiting for new work for container container_1463520580652_0001_00_000002
2016-05-17 21:29:40,953 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520580652_0001_00_000002
2016-05-17 21:29:40,953 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520580652_0001_00_000002
2016-05-17 21:29:40,957 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/2073482192C74A4CA4C23E76685B6C7E/DC68228C10C94D77AAF1BF2476147AED/.tez/application_1463520580652_0001
2016-05-17 21:29:40,958 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_SWAP/_temporary
2016-05-17 21:29:40,958 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.323
2016-05-17 21:29:40,966 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:40,966 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:41,019 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_SWAP
2016-05-17 21:29:41,038 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:41,038 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:41,065 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 21:29:41,078 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-17 21:29:41,078 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:41,079 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:41,079 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:41,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:41,099 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_-1
2016-05-17 21:29:41,141 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:41,141 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:41,141 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_-1"]
2016-05-17 21:29:41,141 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:41,141 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:41,141 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:41,142 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...emblyplatform/0_VALUES_-1
2016-05-17 21:29:41,142 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:41,162 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:41,166 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:41,167 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:41,167 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:41,175 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/86FDA09F0F384F518E2D8D4D2F976709/EE3118A9C3844D87A62659B2D3B216FB/.tez/application_1463520581166_0001 doesn't exist and is created
2016-05-17 21:29:41,190 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:41,191 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:41,191 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/86FDA09F0F384F518E2D8D4D2F976709/EE3118A9C3844D87A62659B2D3B216FB/.tez/application_1463520581166_0001
2016-05-17 21:29:41,194 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520581166_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:41,204 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:41,204 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:41,210 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:41,210 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:41,211 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/86FDA09F0F384F518E2D8D4D2F976709/EE3118A9C3844D87A62659B2D3B216FB/.tez/application_1463520581166_0001/localmode-log-dir
2016-05-17 21:29:41,211 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/86FDA09F0F384F518E2D8D4D2F976709/EE3118A9C3844D87A62659B2D3B216FB/.tez/application_1463520581166_0001/localmode-log-dir/history.txt.appattempt_1463520581166_0001_000000, maxErrors=10
2016-05-17 21:29:41,211 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:41,211 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520581166_0001_000000, appSubmitTime=1463520581193, launchTime=1463520581194
2016-05-17 21:29:41,215 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:41,217 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:41,218 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49924
2016-05-17 21:29:41,219 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:41,219 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:41,221 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:41,223 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520581166_0001_000000, startTime=1463520581222
2016-05-17 21:29:41,223 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:41,223 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:41,291 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:41,291 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:41,291 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...emblyplatform/0_VALUES_-1, applicationId=application_1463520581166_0001, dagName=[86FDA09F0F384F518E2D8D4D2F976709/EE3118A9C3844D87A62659B2D3B216FB] every/(1/1) ...emblyplatform/0_VALUES_-1
2016-05-17 21:29:41,305 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [86FDA09F0F384F518E2D8D4D2F976709/EE3118A9C3844D87A62659B2D3B216FB] every/(1/1) ...emblyplatform/0_VALUES_-1
2016-05-17 21:29:41,306 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520581166_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/86FDA09F0F384F518E2D8D4D2F976709/EE3118A9C3844D87A62659B2D3B216FB/.tez/application_1463520581166_0001/localmode-log-dir/dag_1463520581166_0001_1.dot
2016-05-17 21:29:41,307 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/86FDA09F0F384F518E2D8D4D2F976709/EE3118A9C3844D87A62659B2D3B216FB/.tez/application_1463520581166_0001/localmode-log-dir/dag_1463520581166_0001_1-tez-dag.pb.txt
2016-05-17 21:29:41,312 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520581166_0001_1
2016-05-17 21:29:41,312 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:41,313 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [86FDA09F0F384F518E2D8D4D2F976709/EE3118A9C3844D87A62659B2D3B216FB] every/(1/1) ...emblyplatform/0_VALUES_-1
2016-05-17 21:29:41 Running Dag: dag_1463520581166_0001_1
2016-05-17 21:29:41,323 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520581166_0001_1, submitTime=1463520581305
2016-05-17 21:29:41,323 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:41,324 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F00FDCE745E843FD93E92DE08E0AB0CD
2016-05-17 21:29:41,324 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:41,324 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:41,324 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520581166_0001_1, initTime=1463520581323
2016-05-17 21:29:41,325 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520581166_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:41,325 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:DAG_STARTED]: dagID=dag_1463520581166_0001_1, startTime=1463520581325
2016-05-17 21:29:41,325 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:41,325 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:41,325 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520581166_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:41,325 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BEAE9C3C128D40C08726A5A99AAA023F : {456CA7AEA2A84395962760F788CB8182={InputName=456CA7AEA2A84395962760F788CB8182}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:41,325 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 456CA7AEA2A84395962760F788CB8182, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:41,326 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F]
2016-05-17 21:29:41,326 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F]
2016-05-17 21:29:41,326 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F]
2016-05-17 21:29:41,326 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...emblyplatform/0_VALUES_-1, applicationId=application_1463520581166_0001, dagName=[86FDA09F0F384F518E2D8D4D2F976709/EE3118A9C3844D87A62659B2D3B216FB] every/(1/1) ...emblyplatform/0_VALUES_-1
2016-05-17 21:29:41,326 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F]. Starting root input initializers: 1
2016-05-17 21:29:41,326 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:41,326 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 456CA7AEA2A84395962760F788CB8182 on vertex vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F]
2016-05-17 21:29:41,327 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD]
2016-05-17 21:29:41,328 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:41,328 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 456CA7AEA2A84395962760F788CB8182 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:41,328 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:41,329 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:41,329 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:41,330 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD]
2016-05-17 21:29:41,330 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD]
2016-05-17 21:29:41,330 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD]
2016-05-17 21:29:41,330 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9F49495B56004EC1BFB72896870904D0, vertexId=vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:41,330 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9F49495B56004EC1BFB72896870904D0, vertexId=vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD]
2016-05-17 21:29:41,331 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:41,331 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 456CA7AEA2A84395962760F788CB8182 on vertex vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F]
2016-05-17 21:29:41,332 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9F49495B56004EC1BFB72896870904D0, vertexName=F00FDCE745E843FD93E92DE08E0AB0CD, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:41,332 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9F49495B56004EC1BFB72896870904D0, vertexId=vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD]
2016-05-17 21:29:41,334 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:41,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F00FDCE745E843FD93E92DE08E0AB0CD, vertexId=vertex_1463520581166_0001_1_01, initRequestedTime=1463520581326, initedTime=1463520581333, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:41,334 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:41,334 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:41,334 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {456CA7AEA2A84395962760F788CB8182=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:41,335 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F] parallelism set to 1
2016-05-17 21:29:41,335 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:41,335 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520581166_0001_1_00, eventCount=1
2016-05-17 21:29:41,336 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:41,336 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BEAE9C3C128D40C08726A5A99AAA023F, vertexId=vertex_1463520581166_0001_1_00, initRequestedTime=1463520581325, initedTime=1463520581335, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:41,336 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F] with distanceFromRoot: 0
2016-05-17 21:29:41,336 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:41,336 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BEAE9C3C128D40C08726A5A99AAA023F
2016-05-17 21:29:41,337 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F]
2016-05-17 21:29:41,337 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:41,337 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520581166_0001_1_00, startRequestedTime=1463520581336, startedTime=1463520581336
2016-05-17 21:29:41,337 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:41,338 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:41,338 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:TASK_STARTED]: vertexName=BEAE9C3C128D40C08726A5A99AAA023F, taskId=task_1463520581166_0001_1_00_000000, scheduledTime=1463520581338, launchTime=1463520581338
2016-05-17 21:29:41,338 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520581166_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:41,338 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520581166_0001_1_00 for vertex: vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD] numStartedSources: 1 numSources: 1
2016-05-17 21:29:41,339 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD] with distanceFromRoot: 1
2016-05-17 21:29:41,339 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F]
2016-05-17 21:29:41,339 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BEAE9C3C128D40C08726A5A99AAA023F in vertex: F00FDCE745E843FD93E92DE08E0AB0CD
2016-05-17 21:29:41,339 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F00FDCE745E843FD93E92DE08E0AB0CD with 1 source tasks and 1 pending tasks
2016-05-17 21:29:41,339 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520581166_0001, with dag id: dag_1463520581166_0001_1
2016-05-17 21:29:41,340 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:41,340 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520581166_0001_1_01, startRequestedTime=1463520581339, startedTime=1463520581339
2016-05-17 21:29:41,340 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:41,340 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [86FDA09F0F384F518E2D8D4D2F976709/EE3118A9C3844D87A62659B2D3B216FB] every/(1/1) ...emblyplatform/0_VALUES_-1, VertexName: BEAE9C3C128D40C08726A5A99AAA023F, VertexParallelism: 1, TaskAttemptID:attempt_1463520581166_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=456CA7AEA2A84395962760F788CB8182, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F00FDCE745E843FD93E92DE08E0AB0CD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:41,340 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520581166_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:41,340 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:41,341 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:41,341 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:41,342 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:41,342 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520581166_0001_00_000001
2016-05-17 21:29:41,342 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581166_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:41,343 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520581166_0001_00_000001
2016-05-17 21:29:41,343 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:41,344 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581166_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:41,344 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:41,344 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520581166_0001_00_000001, launchTime=1463520581344
2016-05-17 21:29:41,345 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520581166_0001_00_000001
2016-05-17 21:29:41,345 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520581166_0001_1_00_000000_0] to container: [container_1463520581166_0001_00_000001]
2016-05-17 21:29:41,345 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581166_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:41,346 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520581166_0001_1_00_000000_0] started. Is using containerId: [container_1463520581166_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:41,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BEAE9C3C128D40C08726A5A99AAA023F, taskAttemptId=attempt_1463520581166_0001_1_00_000000_0, startTime=1463520581346, containerId=container_1463520581166_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520581166_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:41,346 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520581166_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:41,346 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520581166_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:41,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:41,346 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520581166_0001_00_000001 given task: attempt_1463520581166_0001_1_00_000000_0
2016-05-17 21:29:41,347 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520581166_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520581166_0001_1_00_000000_0
2016-05-17 21:29:41 Starting to run new task attempt: attempt_1463520581166_0001_1_00_000000_0
2016-05-17 21:29:41,347 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520581166_0001_1_00_000000_0
2016-05-17 21:29:41,347 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:41,347 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:41,347 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:41,347 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:41,347 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:41,347 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:41,347 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [86FDA09F0F384F518E2D8D4D2F976709/EE3118A9C3844D87A62659B2D3B216FB] every/(1/1) ...emblyplatform/0_VALUES_-1, VertexName: BEAE9C3C128D40C08726A5A99AAA023F, VertexParallelism: 1, TaskAttemptID:attempt_1463520581166_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=456CA7AEA2A84395962760F788CB8182, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F00FDCE745E843FD93E92DE08E0AB0CD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:41,348 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:41,348 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520581166_0001_1_00_000000_0
2016-05-17 21:29:41,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:41,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:41,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:41,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=456CA7AEA2A84395962760F788CB8182, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:41,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:41,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 456CA7AEA2A84395962760F788CB8182
2016-05-17 21:29:41,349 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 456CA7AEA2A84395962760F788CB8182
2016-05-17 21:29:41,350 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520581166_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:41,351 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F00FDCE745E843FD93E92DE08E0AB0CD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:41,351 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:41,351 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:41,351 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 456CA7AEA2A84395962760F788CB8182
2016-05-17 21:29:41,352 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 456CA7AEA2A84395962760F788CB8182
2016-05-17 21:29:41,351 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:41,352 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F00FDCE745E843FD93E92DE08E0AB0CD
2016-05-17 21:29:41,354 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:41,354 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F00FDCE745E843FD93E92DE08E0AB0CD
2016-05-17 21:29:41,354 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F00FDCE745E843FD93E92DE08E0AB0CD
2016-05-17 21:29:41,354 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BEAE9C3C128D40C08726A5A99AAA023F, ordinal: 0
2016-05-17 21:29:41,354 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEAE9C3C128D40C08726A5A99AAA023F, mem on start (mb), free: 277, total: 361, max: 672
2016-05-17 21:29:41,354 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:41,354 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:41,355 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:41,355 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:41,355 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:41,355 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:41,355 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:41,355 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:41,355 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:41,355 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:41,355 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 456CA7AEA2A84395962760F788CB8182, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:41,356 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F00FDCE745E843FD93E92DE08E0AB0CD, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:41,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:41,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 456CA7AEA2A84395962760F788CB8182 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:41,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:41,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:41,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 456CA7AEA2A84395962760F788CB8182
2016-05-17 21:29:41,357 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:41,357 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 456CA7AEA2A84395962760F788CB8182
2016-05-17 21:29:41,357 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:41,358 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520581166_0001_1_00_000000_0
2016-05-17 21:29:41,358 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:41,358 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@240864bd
2016-05-17 21:29:41,359 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@240864bd
2016-05-17 21:29:41,359 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:41,360 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:41,360 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:41,361 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 456CA7AEA2A84395962760F788CB8182
2016-05-17 21:29:41,361 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: B90385943D044B079A664F83D3F31087
2016-05-17 21:29:41,361 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] B90385943D044B079A664F83D3F31087
2016-05-17 21:29:41,362 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:41,364 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:41,364 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3f65b609; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@2e922963; comparator=cascading.tuple.hadoop.util.TupleComparator@7ddb3a77; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:41,364 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:41,364 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:41,365 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:41,378 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:41,378 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:41,379 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:41,379 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:41,379 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:41,379 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:41,380 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:41,380 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BEAE9C3C128D40C08726A5A99AAA023F, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:41,384 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520581166_0001_1_00_000000_0
2016-05-17 21:29:41,384 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEAE9C3C128D40C08726A5A99AAA023F, mem on close (mb), free: 174, total: 361, max: 672
2016-05-17 21:29:41,384 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:41,384 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:41,384 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:41,384 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:41,387 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:41 Completed running task attempt: attempt_1463520581166_0001_1_00_000000_0
2016-05-17 21:29:41,390 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520581166_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:41,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379060224, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520581380, Process_Duration=4, Process_End_Time=1463520581384, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:41,390 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520581166_0001_1_00, eventCount=1
2016-05-17 21:29:41,390 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:41,390 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:41,391 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520581166_0001_00_000001
2016-05-17 21:29:41,391 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BEAE9C3C128D40C08726A5A99AAA023F, taskAttemptId=attempt_1463520581166_0001_1_00_000000_0, startTime=1463520581346, finishTime=1463520581391, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379060224, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520581380, Process_Duration=4, Process_End_Time=1463520581384, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:41,391 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520581166_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:41,391 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:41,391 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:41,391 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520581166_0001_1_00_000000_0
2016-05-17 21:29:41,392 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:41,392 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:TASK_FINISHED]: vertexName=BEAE9C3C128D40C08726A5A99AAA023F, taskId=task_1463520581166_0001_1_00_000000, startTime=1463520581346, finishTime=1463520581392, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1463520581166_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379060224, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520581380, Process_Duration=4, Process_End_Time=1463520581384, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:41,393 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520581166_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:41,393 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F] : 1
2016-05-17 21:29:41,393 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F]
2016-05-17 21:29:41,393 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:41,393 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:VERTEX_FINISHED]: vertexName=BEAE9C3C128D40C08726A5A99AAA023F, vertexId=vertex_1463520581166_0001_1_00, initRequestedTime=1463520581325, initedTime=1463520581335, startRequestedTime=1463520581336, startedTime=1463520581336, finishTime=1463520581393, timeTaken=57, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379060224, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520581380, Process_Duration=4, Process_End_Time=1463520581384, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520581346, firstTasksToStart=[ task_1463520581166_0001_1_00_000000 ], lastTaskFinishTime=1463520581391, lastTasksToFinish=[ task_1463520581166_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520581166_0001_1_00_000000 ], longestDurationTasks=[ task_1463520581166_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:41,394 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:41,394 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581166_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:41,394 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520581166_0001_00_000001, stoppedTime=1463520581394, exitStatus=0
2016-05-17 21:29:41,394 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581166_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:41,394 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:41,394 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD] attempt: attempt_1463520581166_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:41,394 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F00FDCE745E843FD93E92DE08E0AB0CD
2016-05-17 21:29:41,394 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD]
2016-05-17 21:29:41,395 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:41,395 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520581166_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:41,395 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:TASK_STARTED]: vertexName=F00FDCE745E843FD93E92DE08E0AB0CD, taskId=task_1463520581166_0001_1_01_000000, scheduledTime=1463520581395, launchTime=1463520581395
2016-05-17 21:29:41,395 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520581166_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:41,395 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD]
2016-05-17 21:29:41,395 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:41,396 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581166_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:41,396 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [86FDA09F0F384F518E2D8D4D2F976709/EE3118A9C3844D87A62659B2D3B216FB] every/(1/1) ...emblyplatform/0_VALUES_-1, VertexName: F00FDCE745E843FD93E92DE08E0AB0CD, VertexParallelism: 1, TaskAttemptID:attempt_1463520581166_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BEAE9C3C128D40C08726A5A99AAA023F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9F49495B56004EC1BFB72896870904D0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:41,397 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520581166_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:41,397 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:41,397 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520581166_0001_00_000002
2016-05-17 21:29:41,397 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581166_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:41,399 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520581166_0001_00_000002
2016-05-17 21:29:41,399 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:41,399 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581166_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:41,400 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:41,400 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520581166_0001_00_000002, launchTime=1463520581400
2016-05-17 21:29:41,401 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520581166_0001_00_000002
2016-05-17 21:29:41,401 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520581166_0001_1_01_000000_0] to container: [container_1463520581166_0001_00_000002]
2016-05-17 21:29:41,401 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581166_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:41,401 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520581166_0001_1_01_000000_0] started. Is using containerId: [container_1463520581166_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:41,402 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F00FDCE745E843FD93E92DE08E0AB0CD, taskAttemptId=attempt_1463520581166_0001_1_01_000000_0, startTime=1463520581401, containerId=container_1463520581166_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520581166_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:41,402 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520581166_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:41,402 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520581166_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:41,402 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:41,402 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520581166_0001_00_000002 given task: attempt_1463520581166_0001_1_01_000000_0
2016-05-17 21:29:41,402 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520581166_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520581166_0001_1_01_000000_0
2016-05-17 21:29:41,402 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520581166_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:41,402 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520581166_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:41,402 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520581166_0001_00_000001, Exiting
2016-05-17 21:29:41,402 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520581166_0001_00_000001
2016-05-17 21:29:41,403 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520581166_0001_00_000001
2016-05-17 21:29:41 Starting to run new task attempt: attempt_1463520581166_0001_1_01_000000_0
2016-05-17 21:29:41,403 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520581166_0001_1_01_000000_0
2016-05-17 21:29:41,403 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:41,403 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:41,403 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:41,408 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520581166_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:41,408 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520581166_0001_00_000001 completed successfully
2016-05-17 21:29:41,408 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581166_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:41,410 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:41,410 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:41,410 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:41,411 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [86FDA09F0F384F518E2D8D4D2F976709/EE3118A9C3844D87A62659B2D3B216FB] every/(1/1) ...emblyplatform/0_VALUES_-1, VertexName: F00FDCE745E843FD93E92DE08E0AB0CD, VertexParallelism: 1, TaskAttemptID:attempt_1463520581166_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BEAE9C3C128D40C08726A5A99AAA023F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9F49495B56004EC1BFB72896870904D0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:41,411 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:41,412 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520581166_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:41,413 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520581166_0001_1_01_000000_0
2016-05-17 21:29:41,413 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:41,413 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:41,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:41,416 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:41,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9F49495B56004EC1BFB72896870904D0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:41,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:41,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9F49495B56004EC1BFB72896870904D0
2016-05-17 21:29:41,418 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:41,418 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9F49495B56004EC1BFB72896870904D0
2016-05-17 21:29:41,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BEAE9C3C128D40C08726A5A99AAA023F, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:41,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:41,419 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BEAE9C3C128D40C08726A5A99AAA023F
2016-05-17 21:29:41,421 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:41,421 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:41,421 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BEAE9C3C128D40C08726A5A99AAA023F
2016-05-17 21:29:41,421 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BEAE9C3C128D40C08726A5A99AAA023F
2016-05-17 21:29:41,423 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F00FDCE745E843FD93E92DE08E0AB0CD, ordinal: 1
2016-05-17 21:29:41,423 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F00FDCE745E843FD93E92DE08E0AB0CD, mem on start (mb), free: 168, total: 361, max: 672
2016-05-17 21:29:41,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:41,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:41,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:41,429 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:41,429 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9F49495B56004EC1BFB72896870904D0
2016-05-17 21:29:41,429 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:41,430 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:41,430 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:41,430 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:41,431 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:41,431 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:41,431 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:41,431 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9F49495B56004EC1BFB72896870904D0, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:41,431 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BEAE9C3C128D40C08726A5A99AAA023F, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:41,431 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:41,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BEAE9C3C128D40C08726A5A99AAA023F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:41,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BEAE9C3C128D40C08726A5A99AAA023F
2016-05-17 21:29:41,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:41,433 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:41,434 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:41,434 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:41,434 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BEAE9C3C128D40C08726A5A99AAA023F with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:41,434 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:41,434 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:41,435 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:41,435 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BEAE9C3C128D40C08726A5A99AAA023F
2016-05-17 21:29:41,435 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:41,437 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520581166_0001_1_00_000000_0_10177, runDuration: 36413, ]
2016-05-17 21:29:41,437 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520581166_0001_1_01_000000_0
2016-05-17 21:29:41,440 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:41,440 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: B90385943D044B079A664F83D3F31087
2016-05-17 21:29:41,440 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_-1"], id: 9F49495B56004EC1BFB72896870904D0
2016-05-17 21:29:41,441 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_-1"]
2016-05-17 21:29:41,441 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520581166_0001_1_00_000000_0_10177] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520581441, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:41,441 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BEAE9C3C128D40C08726A5A99AAA023F
2016-05-17 21:29:41,441 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] B90385943D044B079A664F83D3F31087, for 1 inputs
2016-05-17 21:29:41,441 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:41,441 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [BEAE9C3C128D40C08726A5A99AAA023F] #30 in 1ms
2016-05-17 21:29:41,442 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BEAE9C3C128D40C08726A5A99AAA023F] #30, status:false, isInterrupted:false
2016-05-17 21:29:41,442 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:41,443 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:41,443 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:41,443 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:41,444 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:41,444 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:41,444 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BEAE9C3C128D40C08726A5A99AAA023F
2016-05-17 21:29:41,445 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:41,444 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F00FDCE745E843FD93E92DE08E0AB0CD, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:41,446 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:41,447 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@a8e76ee; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@f2208ac
2016-05-17 21:29:41,448 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:41,448 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:41,448 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:41,448 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:41,449 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520581166_0001_1_01_000000_0
2016-05-17 21:29:41,450 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520581166_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:41,451 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520581166_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:41,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F00FDCE745E843FD93E92DE08E0AB0CD, taskAttemptId=attempt_1463520581166_0001_1_01_000000_0, startTime=1463520581401, finishTime=1463520581452, timeTaken=51, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379060224, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520581445, Process_Duration=6, Process_End_Time=1463520581451, Read_Duration=1, Tuples_Read=1
2016-05-17 21:29:41,453 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520581166_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:41,453 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520581166_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:41,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:41,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:TASK_FINISHED]: vertexName=F00FDCE745E843FD93E92DE08E0AB0CD, taskId=task_1463520581166_0001_1_01_000000, startTime=1463520581401, finishTime=1463520581453, timeTaken=52, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:41,453 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520581166_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:41,453 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:41,453 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD] : 1
2016-05-17 21:29:41,453 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD] because task failed: task_1463520581166_0001_1_01_000000
2016-05-17 21:29:41,454 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:41,454 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:41,454 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD]
2016-05-17 21:29:41,454 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=9F49495B56004EC1BFB72896870904D0, vertexId=vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD]
2016-05-17 21:29:41,455 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:VERTEX_FINISHED]: vertexName=F00FDCE745E843FD93E92DE08E0AB0CD, vertexId=vertex_1463520581166_0001_1_01, initRequestedTime=1463520581326, initedTime=1463520581333, startRequestedTime=1463520581339, startedTime=1463520581339, finishTime=1463520581455, timeTaken=116, status=FAILED, diagnostics=Task failed, taskId=task_1463520581166_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520581401, firstTasksToStart=[ task_1463520581166_0001_1_01_000000 ], lastTaskFinishTime=1463520581452, lastTasksToFinish=[ task_1463520581166_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:41,455 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:41,454 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:41,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379060224, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520581445, Process_Duration=6, Process_End_Time=1463520581451, Read_Duration=1, Tuples_Read=1]]
2016-05-17 21:29:41,456 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F00FDCE745E843FD93E92DE08E0AB0CD, mem on close (mb), free: 164, total: 361, max: 672
2016-05-17 21:29:41,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=F00FDCE745E843FD93E92DE08E0AB0CD, index=1
2016-05-17 21:29:41,457 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BEAE9C3C128D40C08726A5A99AAA023F
2016-05-17 21:29:41,455 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:41,459 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:41,456 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520581166_0001_00_000002, stoppedTime=1463520581455, exitStatus=0
2016-05-17 21:29:41,460 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581166_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:41,460 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520581166_0001_1_01_000000_0 failed: true container: container_1463520581166_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:41,461 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:41,461 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:41,461 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:41,461 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520581166_0001_1
2016-05-17 21:29:41,461 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 9F49495B56004EC1BFB72896870904D0 for vertex: vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD]
2016-05-17 21:29:41,461 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:41,462 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:41,462 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520581166_0001_1_00 [BEAE9C3C128D40C08726A5A99AAA023F]
2016-05-17 21:29:41,462 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:41,462 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:41,462 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:41,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=F00FDCE745E843FD93E92DE08E0AB0CD, sourceVertex=BEAE9C3C128D40C08726A5A99AAA023F
2016-05-17 21:29:41,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=F00FDCE745E843FD93E92DE08E0AB0CD, sourceVertex=9F49495B56004EC1BFB72896870904D0
2016-05-17 21:29:41,462 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520581166_0001_00_000002
2016-05-17 21:29:41,462 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520581166_0001_00_000002
2016-05-17 21:29:41,463 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520581166_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:41,463 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520581166_0001_00_000002 completed but with errors
2016-05-17 21:29:41,464 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520581166_0001_1, queueSize=0
2016-05-17 21:29:41,464 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:41,465 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581166_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520581166_0001_1, startTime=1463520581325, finishTime=1463520581462, timeTaken=137, status=FAILED, diagnostics=Vertex failed, vertexName=F00FDCE745E843FD93E92DE08E0AB0CD, vertexId=vertex_1463520581166_0001_1_01, diagnostics=[Task failed, taskId=task_1463520581166_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520581166_0001_1_01 [F00FDCE745E843FD93E92DE08E0AB0CD] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379060224, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520581380, Process_Duration=4, Process_End_Time=1463520581384, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:41,465 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520581166_0001_1 finished with state: FAILED
2016-05-17 21:29:41,465 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520581166_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:41,465 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520581166_0001_00_000002
2016-05-17 21:29:41,465 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520581166_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:41,465 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581166_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:41 Completed Dag: dag_1463520581166_0001_1
2016-05-17 21:29:41,466 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520581166_0001_1, dagState=FAILED
2016-05-17 21:29:41,466 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520581166_0001_1_post
2016-05-17 21:29:41,466 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:41,466 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:41,466 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520581166_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:41,475 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...emblyplatform/0_VALUES_-1, applicationId=application_1463520581166_0001
2016-05-17 21:29:41,476 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:41,476 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:41,476 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:41,476 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:41,476 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:41,476 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:41,477 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:41,477 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:41,477 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:41,477 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:41,477 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:41,478 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:41,478 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:41,482 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/86FDA09F0F384F518E2D8D4D2F976709/EE3118A9C3844D87A62659B2D3B216FB/.tez/application_1463520581166_0001
2016-05-17 21:29:41,482 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:41,482 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...emblyplatform/0_VALUES_-1
2016-05-17 21:29:41,483 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:41,483 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_-1/_temporary
2016-05-17 21:29:41,483 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.342
2016-05-17 21:29:41,483 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_-1
2016-05-17 21:29:41,513 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:41,513 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:41,524 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:41,524 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:41,548 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 21:29:41,551 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-17 21:29:41,551 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:41,551 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:41,551 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_'field'
2016-05-17 21:29:41,582 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:41,583 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:41,626 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:29:41,637 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-17 21:29:41,638 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:41,638 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:41,638 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:41,651 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:41,655 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_RESULTS
2016-05-17 21:29:41,698 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:41,699 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:41,699 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_RESULTS"]
2016-05-17 21:29:41,699 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:41,700 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:41,700 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:41,701 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...platform/0_VALUES_RESULTS
2016-05-17 21:29:41,701 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:41,713 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:41,718 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:41,718 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:41,718 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:41,729 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AA443529417144A3A3E7F51FA1C85905/E01FCBA356754FA89FE6D38DAE03E1C2/.tez/application_1463520581718_0001 doesn't exist and is created
2016-05-17 21:29:41,746 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:41,746 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:41,746 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AA443529417144A3A3E7F51FA1C85905/E01FCBA356754FA89FE6D38DAE03E1C2/.tez/application_1463520581718_0001
2016-05-17 21:29:41,749 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520581718_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:41,763 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:41,764 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:41,765 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:41,766 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:41,766 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AA443529417144A3A3E7F51FA1C85905/E01FCBA356754FA89FE6D38DAE03E1C2/.tez/application_1463520581718_0001/localmode-log-dir
2016-05-17 21:29:41,766 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AA443529417144A3A3E7F51FA1C85905/E01FCBA356754FA89FE6D38DAE03E1C2/.tez/application_1463520581718_0001/localmode-log-dir/history.txt.appattempt_1463520581718_0001_000000, maxErrors=10
2016-05-17 21:29:41,766 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:41,766 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520581718_0001_000000, appSubmitTime=1463520581749, launchTime=1463520581749
2016-05-17 21:29:41,780 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:41,785 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:41,786 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:41,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:41,792 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:53498
2016-05-17 21:29:41,794 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:41,795 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520581718_0001_000000, startTime=1463520581795
2016-05-17 21:29:41,795 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:41,796 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:41,846 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:41,846 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:41,846 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_RESULTS, applicationId=application_1463520581718_0001, dagName=[AA443529417144A3A3E7F51FA1C85905/E01FCBA356754FA89FE6D38DAE03E1C2] every/(1/1) ...platform/0_VALUES_RESULTS
2016-05-17 21:29:41,862 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [AA443529417144A3A3E7F51FA1C85905/E01FCBA356754FA89FE6D38DAE03E1C2] every/(1/1) ...platform/0_VALUES_RESULTS
2016-05-17 21:29:41,863 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520581718_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AA443529417144A3A3E7F51FA1C85905/E01FCBA356754FA89FE6D38DAE03E1C2/.tez/application_1463520581718_0001/localmode-log-dir/dag_1463520581718_0001_1.dot
2016-05-17 21:29:41,864 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AA443529417144A3A3E7F51FA1C85905/E01FCBA356754FA89FE6D38DAE03E1C2/.tez/application_1463520581718_0001/localmode-log-dir/dag_1463520581718_0001_1-tez-dag.pb.txt
2016-05-17 21:29:41,869 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520581718_0001_1
2016-05-17 21:29:41,869 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:41,869 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [AA443529417144A3A3E7F51FA1C85905/E01FCBA356754FA89FE6D38DAE03E1C2] every/(1/1) ...platform/0_VALUES_RESULTS
2016-05-17 21:29:41 Running Dag: dag_1463520581718_0001_1
2016-05-17 21:29:41,879 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520581718_0001_1, submitTime=1463520581862
2016-05-17 21:29:41,879 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:41,880 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EFC80A4156EB47BBABDF3980A304EE93
2016-05-17 21:29:41,880 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:41,880 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520581718_0001_1, initTime=1463520581879
2016-05-17 21:29:41,880 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:41,881 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520581718_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:41,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:DAG_STARTED]: dagID=dag_1463520581718_0001_1, startTime=1463520581881
2016-05-17 21:29:41,881 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:41,881 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:41,881 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520581718_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:41,881 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D1C7269C5EE84DD9BA5DE553B3C0487C : {49E6D497A2DD4948B5BF8535D07B4C7B={InputName=49E6D497A2DD4948B5BF8535D07B4C7B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:41,881 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 49E6D497A2DD4948B5BF8535D07B4C7B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:41,881 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C]
2016-05-17 21:29:41,882 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C]
2016-05-17 21:29:41,882 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C]
2016-05-17 21:29:41,882 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C]. Starting root input initializers: 1
2016-05-17 21:29:41,882 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:41,882 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93]
2016-05-17 21:29:41,883 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_RESULTS, applicationId=application_1463520581718_0001, dagName=[AA443529417144A3A3E7F51FA1C85905/E01FCBA356754FA89FE6D38DAE03E1C2] every/(1/1) ...platform/0_VALUES_RESULTS
2016-05-17 21:29:41,884 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:41,884 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93]
2016-05-17 21:29:41,884 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 49E6D497A2DD4948B5BF8535D07B4C7B on vertex vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C]
2016-05-17 21:29:41,885 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93]
2016-05-17 21:29:41,885 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93]
2016-05-17 21:29:41,885 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6F9495998B734914B0C95A5FB3E55BCD, vertexId=vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:41,885 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6F9495998B734914B0C95A5FB3E55BCD, vertexId=vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93]
2016-05-17 21:29:41,886 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:41,886 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 49E6D497A2DD4948B5BF8535D07B4C7B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:41,887 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:41,887 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6F9495998B734914B0C95A5FB3E55BCD, vertexName=EFC80A4156EB47BBABDF3980A304EE93, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:41,887 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:41,887 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6F9495998B734914B0C95A5FB3E55BCD, vertexId=vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93]
2016-05-17 21:29:41,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EFC80A4156EB47BBABDF3980A304EE93, vertexId=vertex_1463520581718_0001_1_01, initRequestedTime=1463520581882, initedTime=1463520581888, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:41,888 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:41,888 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:41,888 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:41,888 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 49E6D497A2DD4948B5BF8535D07B4C7B on vertex vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C]
2016-05-17 21:29:41,889 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:41,889 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {49E6D497A2DD4948B5BF8535D07B4C7B=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:41,889 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C] parallelism set to 1
2016-05-17 21:29:41,889 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:41,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520581718_0001_1_00, eventCount=1
2016-05-17 21:29:41,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:41,890 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D1C7269C5EE84DD9BA5DE553B3C0487C, vertexId=vertex_1463520581718_0001_1_00, initRequestedTime=1463520581881, initedTime=1463520581889, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:41,890 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C] with distanceFromRoot: 0
2016-05-17 21:29:41,890 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:41,891 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D1C7269C5EE84DD9BA5DE553B3C0487C
2016-05-17 21:29:41,891 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C]
2016-05-17 21:29:41,891 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:41,892 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520581718_0001_1_00, startRequestedTime=1463520581891, startedTime=1463520581891
2016-05-17 21:29:41,892 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:41,892 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:41,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:TASK_STARTED]: vertexName=D1C7269C5EE84DD9BA5DE553B3C0487C, taskId=task_1463520581718_0001_1_00_000000, scheduledTime=1463520581892, launchTime=1463520581892
2016-05-17 21:29:41,893 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520581718_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:41,893 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520581718_0001_1_00 for vertex: vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93] numStartedSources: 1 numSources: 1
2016-05-17 21:29:41,893 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93] with distanceFromRoot: 1
2016-05-17 21:29:41,894 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C]
2016-05-17 21:29:41,894 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D1C7269C5EE84DD9BA5DE553B3C0487C in vertex: EFC80A4156EB47BBABDF3980A304EE93
2016-05-17 21:29:41,894 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: EFC80A4156EB47BBABDF3980A304EE93 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:41,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:41,895 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520581718_0001_1_01, startRequestedTime=1463520581894, startedTime=1463520581894
2016-05-17 21:29:41,895 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:41,895 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [AA443529417144A3A3E7F51FA1C85905/E01FCBA356754FA89FE6D38DAE03E1C2] every/(1/1) ...platform/0_VALUES_RESULTS, VertexName: D1C7269C5EE84DD9BA5DE553B3C0487C, VertexParallelism: 1, TaskAttemptID:attempt_1463520581718_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=49E6D497A2DD4948B5BF8535D07B4C7B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EFC80A4156EB47BBABDF3980A304EE93, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:41,896 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520581718_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:41,896 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:41,896 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:41,897 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:41,897 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:41,897 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520581718_0001_00_000001
2016-05-17 21:29:41,897 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581718_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:41,899 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520581718_0001_00_000001
2016-05-17 21:29:41,899 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:41,899 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520581718_0001, with dag id: dag_1463520581718_0001_1
2016-05-17 21:29:41,899 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581718_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:41,899 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:41,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520581718_0001_00_000001, launchTime=1463520581899
2016-05-17 21:29:41,900 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520581718_0001_00_000001
2016-05-17 21:29:41,901 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520581718_0001_1_00_000000_0] to container: [container_1463520581718_0001_00_000001]
2016-05-17 21:29:41,901 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581718_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:41,901 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520581718_0001_1_00_000000_0] started. Is using containerId: [container_1463520581718_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:41,901 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D1C7269C5EE84DD9BA5DE553B3C0487C, taskAttemptId=attempt_1463520581718_0001_1_00_000000_0, startTime=1463520581901, containerId=container_1463520581718_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520581718_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:41,901 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520581718_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:41,901 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520581718_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:41,901 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:41,902 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520581718_0001_00_000001 given task: attempt_1463520581718_0001_1_00_000000_0
2016-05-17 21:29:41,902 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520581718_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520581718_0001_1_00_000000_0
2016-05-17 21:29:41 Starting to run new task attempt: attempt_1463520581718_0001_1_00_000000_0
2016-05-17 21:29:41,902 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520581718_0001_1_00_000000_0
2016-05-17 21:29:41,902 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:41,902 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:41,902 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:41,902 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:41,902 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:41,902 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:41,903 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AA443529417144A3A3E7F51FA1C85905/E01FCBA356754FA89FE6D38DAE03E1C2] every/(1/1) ...platform/0_VALUES_RESULTS, VertexName: D1C7269C5EE84DD9BA5DE553B3C0487C, VertexParallelism: 1, TaskAttemptID:attempt_1463520581718_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=49E6D497A2DD4948B5BF8535D07B4C7B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EFC80A4156EB47BBABDF3980A304EE93, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:41,903 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:41,903 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520581718_0001_1_00_000000_0
2016-05-17 21:29:41,903 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520581718_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:41,903 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:41,904 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:41,904 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=49E6D497A2DD4948B5BF8535D07B4C7B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:41,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:41,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 49E6D497A2DD4948B5BF8535D07B4C7B
2016-05-17 21:29:41,904 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EFC80A4156EB47BBABDF3980A304EE93, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:41,904 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:41,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:41,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EFC80A4156EB47BBABDF3980A304EE93
2016-05-17 21:29:41,907 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:41,907 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EFC80A4156EB47BBABDF3980A304EE93
2016-05-17 21:29:41,907 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EFC80A4156EB47BBABDF3980A304EE93
2016-05-17 21:29:41,905 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 49E6D497A2DD4948B5BF8535D07B4C7B
2016-05-17 21:29:41,908 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:41,909 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:41,909 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 49E6D497A2DD4948B5BF8535D07B4C7B
2016-05-17 21:29:41,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 49E6D497A2DD4948B5BF8535D07B4C7B
2016-05-17 21:29:41,911 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D1C7269C5EE84DD9BA5DE553B3C0487C, ordinal: 0
2016-05-17 21:29:41,911 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D1C7269C5EE84DD9BA5DE553B3C0487C, mem on start (mb), free: 285, total: 362, max: 672
2016-05-17 21:29:41,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:41,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:41,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:41,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:41,912 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:41,912 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:41,912 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:41,912 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:41,912 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:41,912 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:41,912 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EFC80A4156EB47BBABDF3980A304EE93, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:41,912 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 49E6D497A2DD4948B5BF8535D07B4C7B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:41,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:41,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 49E6D497A2DD4948B5BF8535D07B4C7B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:41,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:41,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:41,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 49E6D497A2DD4948B5BF8535D07B4C7B
2016-05-17 21:29:41,913 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:41,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 49E6D497A2DD4948B5BF8535D07B4C7B
2016-05-17 21:29:41,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:41,914 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520581718_0001_1_00_000000_0
2016-05-17 21:29:41,914 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:41,915 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@43f76960
2016-05-17 21:29:41,915 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@43f76960
2016-05-17 21:29:41,916 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:41,916 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:41,917 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:41,917 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 49E6D497A2DD4948B5BF8535D07B4C7B
2016-05-17 21:29:41,918 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: CCF953AD3FF14750AE29EEFFB15AA429
2016-05-17 21:29:41,918 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] CCF953AD3FF14750AE29EEFFB15AA429
2016-05-17 21:29:41,919 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:41,921 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:41,921 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@7ba606ec; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@5110af5f; comparator=cascading.tuple.hadoop.util.TupleComparator@1e69af8d; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:41,921 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:41,921 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:41,921 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:41,934 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:41,934 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:41,934 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:41,934 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:41,934 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:41,936 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:41,936 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:41,936 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D1C7269C5EE84DD9BA5DE553B3C0487C, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:41,938 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520581718_0001_1_00_000000_0
2016-05-17 21:29:41,938 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D1C7269C5EE84DD9BA5DE553B3C0487C, mem on close (mb), free: 183, total: 362, max: 672
2016-05-17 21:29:41,938 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:41,939 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:41,939 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:41,939 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:41,942 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:41 Completed running task attempt: attempt_1463520581718_0001_1_00_000000_0
2016-05-17 21:29:41,945 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520581718_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:41,945 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520581718_0001_1_00, eventCount=1
2016-05-17 21:29:41,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D1C7269C5EE84DD9BA5DE553B3C0487C, taskAttemptId=attempt_1463520581718_0001_1_00_000000_0, startTime=1463520581901, finishTime=1463520581945, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379584512, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520581936, Process_Duration=2, Process_End_Time=1463520581938, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:41,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:41,946 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520581718_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:41,946 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:41,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379584512, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520581936, Process_Duration=2, Process_End_Time=1463520581938, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:41,946 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520581718_0001_1_00_000000_0
2016-05-17 21:29:41,947 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:41,947 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:41,947 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:41,947 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:TASK_FINISHED]: vertexName=D1C7269C5EE84DD9BA5DE553B3C0487C, taskId=task_1463520581718_0001_1_00_000000, startTime=1463520581901, finishTime=1463520581947, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1463520581718_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379584512, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520581936, Process_Duration=2, Process_End_Time=1463520581938, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:41,947 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520581718_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:41,947 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C] : 1
2016-05-17 21:29:41,947 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C]
2016-05-17 21:29:41,948 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520581718_0001_00_000001
2016-05-17 21:29:41,948 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:41,948 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:VERTEX_FINISHED]: vertexName=D1C7269C5EE84DD9BA5DE553B3C0487C, vertexId=vertex_1463520581718_0001_1_00, initRequestedTime=1463520581881, initedTime=1463520581889, startRequestedTime=1463520581891, startedTime=1463520581891, finishTime=1463520581947, timeTaken=56, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379584512, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520581936, Process_Duration=2, Process_End_Time=1463520581938, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520581901, firstTasksToStart=[ task_1463520581718_0001_1_00_000000 ], lastTaskFinishTime=1463520581945, lastTasksToFinish=[ task_1463520581718_0001_1_00_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520581718_0001_1_00_000000 ], longestDurationTasks=[ task_1463520581718_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:41,948 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:41,948 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581718_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:41,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520581718_0001_00_000001, stoppedTime=1463520581949, exitStatus=0
2016-05-17 21:29:41,949 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581718_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:41,949 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93] attempt: attempt_1463520581718_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:41,949 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EFC80A4156EB47BBABDF3980A304EE93
2016-05-17 21:29:41,949 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93]
2016-05-17 21:29:41,949 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:41,949 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520581718_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:41,950 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:TASK_STARTED]: vertexName=EFC80A4156EB47BBABDF3980A304EE93, taskId=task_1463520581718_0001_1_01_000000, scheduledTime=1463520581949, launchTime=1463520581949
2016-05-17 21:29:41,950 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520581718_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:41,950 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93]
2016-05-17 21:29:41,950 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581718_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:41,950 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [AA443529417144A3A3E7F51FA1C85905/E01FCBA356754FA89FE6D38DAE03E1C2] every/(1/1) ...platform/0_VALUES_RESULTS, VertexName: EFC80A4156EB47BBABDF3980A304EE93, VertexParallelism: 1, TaskAttemptID:attempt_1463520581718_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D1C7269C5EE84DD9BA5DE553B3C0487C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6F9495998B734914B0C95A5FB3E55BCD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:41,950 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520581718_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:41,950 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:41,950 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:41,950 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:41,951 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520581718_0001_00_000002
2016-05-17 21:29:41,951 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581718_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:41,952 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520581718_0001_00_000002
2016-05-17 21:29:41,953 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:41,953 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581718_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:41,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:41,953 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520581718_0001_00_000002, launchTime=1463520581953
2016-05-17 21:29:41,954 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520581718_0001_00_000002
2016-05-17 21:29:41,954 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520581718_0001_1_01_000000_0] to container: [container_1463520581718_0001_00_000002]
2016-05-17 21:29:41,954 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581718_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:41,955 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520581718_0001_1_01_000000_0] started. Is using containerId: [container_1463520581718_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:41,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EFC80A4156EB47BBABDF3980A304EE93, taskAttemptId=attempt_1463520581718_0001_1_01_000000_0, startTime=1463520581954, containerId=container_1463520581718_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520581718_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:41,955 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520581718_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:41,955 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520581718_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:41,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:41,955 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520581718_0001_00_000002 given task: attempt_1463520581718_0001_1_01_000000_0
2016-05-17 21:29:41,956 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520581718_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520581718_0001_1_01_000000_0
2016-05-17 21:29:41 Starting to run new task attempt: attempt_1463520581718_0001_1_01_000000_0
2016-05-17 21:29:41,956 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520581718_0001_1_01_000000_0
2016-05-17 21:29:41,957 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:41,957 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:41,957 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:41,957 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:41,957 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:41,957 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:41,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AA443529417144A3A3E7F51FA1C85905/E01FCBA356754FA89FE6D38DAE03E1C2] every/(1/1) ...platform/0_VALUES_RESULTS, VertexName: EFC80A4156EB47BBABDF3980A304EE93, VertexParallelism: 1, TaskAttemptID:attempt_1463520581718_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D1C7269C5EE84DD9BA5DE553B3C0487C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6F9495998B734914B0C95A5FB3E55BCD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:41,958 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:41,958 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520581718_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:41,958 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520581718_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:41,958 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520581718_0001_1_01_000000_0
2016-05-17 21:29:41,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:41,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:41,958 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520581718_0001_00_000001, Exiting
2016-05-17 21:29:41,958 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520581718_0001_00_000001
2016-05-17 21:29:41,959 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520581718_0001_00_000001
2016-05-17 21:29:41,959 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520581718_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:41,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:41,962 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:41,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6F9495998B734914B0C95A5FB3E55BCD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:41,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D1C7269C5EE84DD9BA5DE553B3C0487C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:41,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:41,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6F9495998B734914B0C95A5FB3E55BCD
2016-05-17 21:29:41,963 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:41,963 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6F9495998B734914B0C95A5FB3E55BCD
2016-05-17 21:29:41,964 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520581718_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:41,964 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520581718_0001_00_000001 completed successfully
2016-05-17 21:29:41,964 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581718_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:41,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:41,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D1C7269C5EE84DD9BA5DE553B3C0487C
2016-05-17 21:29:41,967 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EFC80A4156EB47BBABDF3980A304EE93, ordinal: 1
2016-05-17 21:29:41,967 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EFC80A4156EB47BBABDF3980A304EE93, mem on start (mb), free: 177, total: 362, max: 672
2016-05-17 21:29:41,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:41,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:41,968 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:41,968 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:41,969 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D1C7269C5EE84DD9BA5DE553B3C0487C
2016-05-17 21:29:41,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D1C7269C5EE84DD9BA5DE553B3C0487C
2016-05-17 21:29:41,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:41,973 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:41,973 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6F9495998B734914B0C95A5FB3E55BCD
2016-05-17 21:29:41,973 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:41,973 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:41,974 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:41,974 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:41,974 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:41,974 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:41,974 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:41,974 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6F9495998B734914B0C95A5FB3E55BCD, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:41,974 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D1C7269C5EE84DD9BA5DE553B3C0487C, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:41,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:41,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D1C7269C5EE84DD9BA5DE553B3C0487C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:41,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:41,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:41,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D1C7269C5EE84DD9BA5DE553B3C0487C
2016-05-17 21:29:41,975 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:41,976 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:41,976 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D1C7269C5EE84DD9BA5DE553B3C0487C with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:41,976 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:41,976 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:41,976 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:41,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D1C7269C5EE84DD9BA5DE553B3C0487C
2016-05-17 21:29:41,979 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:41,979 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520581718_0001_1_01_000000_0
2016-05-17 21:29:41,981 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520581718_0001_1_00_000000_0_10183, runDuration: 38062, ]
2016-05-17 21:29:41,982 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520581718_0001_1_00_000000_0_10183] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520581982, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:41,982 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D1C7269C5EE84DD9BA5DE553B3C0487C
2016-05-17 21:29:41,982 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:41,982 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:41,982 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D1C7269C5EE84DD9BA5DE553B3C0487C] #31 in 1ms
2016-05-17 21:29:41,983 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D1C7269C5EE84DD9BA5DE553B3C0487C] #31, status:false, isInterrupted:false
2016-05-17 21:29:41,983 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: CCF953AD3FF14750AE29EEFFB15AA429
2016-05-17 21:29:41,983 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:41,984 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:41,984 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:41,984 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:41,985 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:41,985 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:41,985 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D1C7269C5EE84DD9BA5DE553B3C0487C
2016-05-17 21:29:41,985 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:41,985 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_RESULTS"], id: 6F9495998B734914B0C95A5FB3E55BCD
2016-05-17 21:29:41,985 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_RESULTS"]
2016-05-17 21:29:41,985 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] CCF953AD3FF14750AE29EEFFB15AA429, for 1 inputs
2016-05-17 21:29:41,986 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EFC80A4156EB47BBABDF3980A304EE93, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:41,987 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:41,988 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@2d34d5e5; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@67c6c492
2016-05-17 21:29:41,990 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:41,990 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:41,990 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:41,990 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:41,991 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520581718_0001_1_01_000000_0
2016-05-17 21:29:41,991 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520581718_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:41,991 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520581718_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:41,992 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:41,993 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EFC80A4156EB47BBABDF3980A304EE93, taskAttemptId=attempt_1463520581718_0001_1_01_000000_0, startTime=1463520581954, finishTime=1463520581992, timeTaken=38, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379584512, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520581986, Process_Duration=5, Process_End_Time=1463520581991, Read_Duration=0, Tuples_Read=1
2016-05-17 21:29:41,993 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520581718_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:41,993 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520581718_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:41,993 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:TASK_FINISHED]: vertexName=EFC80A4156EB47BBABDF3980A304EE93, taskId=task_1463520581718_0001_1_01_000000, startTime=1463520581954, finishTime=1463520581993, timeTaken=39, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:41,993 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:41,993 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520581718_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:41,993 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93] : 1
2016-05-17 21:29:41,994 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93] because task failed: task_1463520581718_0001_1_01_000000
2016-05-17 21:29:41,994 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:41,994 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:41,994 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93]
2016-05-17 21:29:41,994 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=6F9495998B734914B0C95A5FB3E55BCD, vertexId=vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93]
2016-05-17 21:29:41,994 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:41,995 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:41,995 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:VERTEX_FINISHED]: vertexName=EFC80A4156EB47BBABDF3980A304EE93, vertexId=vertex_1463520581718_0001_1_01, initRequestedTime=1463520581882, initedTime=1463520581888, startRequestedTime=1463520581894, startedTime=1463520581894, finishTime=1463520581994, timeTaken=100, status=FAILED, diagnostics=Task failed, taskId=task_1463520581718_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520581954, firstTasksToStart=[ task_1463520581718_0001_1_01_000000 ], lastTaskFinishTime=1463520581992, lastTasksToFinish=[ task_1463520581718_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:41,995 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:41,995 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520581718_0001_00_000002, stoppedTime=1463520581995, exitStatus=0
2016-05-17 21:29:41,995 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581718_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:41,995 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520581718_0001_1_01_000000_0 failed: true container: container_1463520581718_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:41,995 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:41,995 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:41,995 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520581718_0001_1
2016-05-17 21:29:41,996 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 6F9495998B734914B0C95A5FB3E55BCD for vertex: vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93]
2016-05-17 21:29:41,996 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520581718_0001_1_00 [D1C7269C5EE84DD9BA5DE553B3C0487C]
2016-05-17 21:29:41,996 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520581718_0001_1, queueSize=0
2016-05-17 21:29:41,997 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520581718_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520581718_0001_1, startTime=1463520581881, finishTime=1463520581996, timeTaken=115, status=FAILED, diagnostics=Vertex failed, vertexName=EFC80A4156EB47BBABDF3980A304EE93, vertexId=vertex_1463520581718_0001_1_01, diagnostics=[Task failed, taskId=task_1463520581718_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520581718_0001_1_01 [EFC80A4156EB47BBABDF3980A304EE93] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379584512, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520581936, Process_Duration=2, Process_End_Time=1463520581938, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:41,997 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520581718_0001_1 finished with state: FAILED
2016-05-17 21:29:41,997 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520581718_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:41,997 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:41,997 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:41,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379584512, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520581986, Process_Duration=5, Process_End_Time=1463520581991, Read_Duration=0, Tuples_Read=1]]
2016-05-17 21:29:41,997 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EFC80A4156EB47BBABDF3980A304EE93, mem on close (mb), free: 171, total: 362, max: 672
2016-05-17 21:29:41,998 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520581718_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:41,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=EFC80A4156EB47BBABDF3980A304EE93, index=1
2016-05-17 21:29:41 Completed Dag: dag_1463520581718_0001_1
2016-05-17 21:29:41,998 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D1C7269C5EE84DD9BA5DE553B3C0487C
2016-05-17 21:29:41,998 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520581718_0001_1, dagState=FAILED
2016-05-17 21:29:41,998 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520581718_0001_1_post
2016-05-17 21:29:41,999 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:41,999 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:41,999 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:41,999 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581718_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:42,000 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:42,000 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:42,000 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:42,000 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:42,000 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:42,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=EFC80A4156EB47BBABDF3980A304EE93, sourceVertex=D1C7269C5EE84DD9BA5DE553B3C0487C
2016-05-17 21:29:42,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=EFC80A4156EB47BBABDF3980A304EE93, sourceVertex=6F9495998B734914B0C95A5FB3E55BCD
2016-05-17 21:29:42,000 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520581718_0001_00_000002
2016-05-17 21:29:42,000 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520581718_0001_00_000002
2016-05-17 21:29:42,001 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520581718_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:42,001 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520581718_0001_00_000002 completed but with errors
2016-05-17 21:29:42,001 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520581718_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:42,001 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520581718_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:42,001 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520581718_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:42,008 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/0_VALUES_RESULTS, applicationId=application_1463520581718_0001
2016-05-17 21:29:42,009 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:42,009 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:42,009 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:42,009 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:42,010 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:42,010 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:42,010 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:42,010 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:42,010 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:42,010 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:42,010 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:42,011 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:42,011 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:42,015 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/AA443529417144A3A3E7F51FA1C85905/E01FCBA356754FA89FE6D38DAE03E1C2/.tez/application_1463520581718_0001
2016-05-17 21:29:42,016 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:42,016 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...platform/0_VALUES_RESULTS
2016-05-17 21:29:42,016 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:42,017 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_RESULTS/_temporary
2016-05-17 21:29:42,017 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.318
2016-05-17 21:29:42,017 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_RESULTS
2016-05-17 21:29:42,043 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:42,044 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:42,088 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 21:29:42,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-05-17 21:29:42,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:42,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:42,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:42,108 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:42,113 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_ALL
2016-05-17 21:29:42,133 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:42,133 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:42,154 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:42,154 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:42,155 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_ALL"]
2016-05-17 21:29:42,155 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:42,155 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:42,155 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:42,155 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...mblyplatform/0_VALUES_ALL
2016-05-17 21:29:42,155 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:42,166 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:42,170 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:42,170 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:42,171 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:42,180 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CB722D543D444658AC54F32E99C5F08B/B1BDEEB2B54C4A62B95193242146BE8C/.tez/application_1463520582170_0001 doesn't exist and is created
2016-05-17 21:29:42,194 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:42,194 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:42,195 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CB722D543D444658AC54F32E99C5F08B/B1BDEEB2B54C4A62B95193242146BE8C/.tez/application_1463520582170_0001
2016-05-17 21:29:42,197 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520582170_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:42,206 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:42,207 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:42,208 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:42,208 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:42,209 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CB722D543D444658AC54F32E99C5F08B/B1BDEEB2B54C4A62B95193242146BE8C/.tez/application_1463520582170_0001/localmode-log-dir
2016-05-17 21:29:42,209 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CB722D543D444658AC54F32E99C5F08B/B1BDEEB2B54C4A62B95193242146BE8C/.tez/application_1463520582170_0001/localmode-log-dir/history.txt.appattempt_1463520582170_0001_000000, maxErrors=10
2016-05-17 21:29:42,209 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:42,210 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520582170_0001_000000, appSubmitTime=1463520582197, launchTime=1463520582197
2016-05-17 21:29:42,214 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34877
2016-05-17 21:29:42,214 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:42,216 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:42,217 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:42,217 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:42,219 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:42,221 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520582170_0001_000000, startTime=1463520582221
2016-05-17 21:29:42,221 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:42,221 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:42,294 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:42,295 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:42,295 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/0_VALUES_ALL, applicationId=application_1463520582170_0001, dagName=[CB722D543D444658AC54F32E99C5F08B/B1BDEEB2B54C4A62B95193242146BE8C] every/(1/1) ...mblyplatform/0_VALUES_ALL
2016-05-17 21:29:42,309 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [CB722D543D444658AC54F32E99C5F08B/B1BDEEB2B54C4A62B95193242146BE8C] every/(1/1) ...mblyplatform/0_VALUES_ALL
2016-05-17 21:29:42,310 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520582170_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CB722D543D444658AC54F32E99C5F08B/B1BDEEB2B54C4A62B95193242146BE8C/.tez/application_1463520582170_0001/localmode-log-dir/dag_1463520582170_0001_1.dot
2016-05-17 21:29:42,310 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CB722D543D444658AC54F32E99C5F08B/B1BDEEB2B54C4A62B95193242146BE8C/.tez/application_1463520582170_0001/localmode-log-dir/dag_1463520582170_0001_1-tez-dag.pb.txt
2016-05-17 21:29:42,316 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520582170_0001_1
2016-05-17 21:29:42,316 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:42,316 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [CB722D543D444658AC54F32E99C5F08B/B1BDEEB2B54C4A62B95193242146BE8C] every/(1/1) ...mblyplatform/0_VALUES_ALL
2016-05-17 21:29:42 Running Dag: dag_1463520582170_0001_1
2016-05-17 21:29:42,326 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520582170_0001_1, submitTime=1463520582309
2016-05-17 21:29:42,326 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:42,326 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C9ADE6D058B34033AA904C47B080E93D
2016-05-17 21:29:42,326 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:42,327 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520582170_0001_1, initTime=1463520582326
2016-05-17 21:29:42,327 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520582170_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:42,327 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:42,327 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/0_VALUES_ALL, applicationId=application_1463520582170_0001, dagName=[CB722D543D444658AC54F32E99C5F08B/B1BDEEB2B54C4A62B95193242146BE8C] every/(1/1) ...mblyplatform/0_VALUES_ALL
2016-05-17 21:29:42,328 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:DAG_STARTED]: dagID=dag_1463520582170_0001_1, startTime=1463520582328
2016-05-17 21:29:42,329 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:42,328 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:42,329 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520582170_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:42,329 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F06CAEFB6C3C44CCA2EF09508AE9D011 : {7E85923CAEF64820B344F0F42CC27621={InputName=7E85923CAEF64820B344F0F42CC27621}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:42,329 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 7E85923CAEF64820B344F0F42CC27621, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:42,329 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011]
2016-05-17 21:29:42,329 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011]
2016-05-17 21:29:42,329 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011]
2016-05-17 21:29:42,329 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011]. Starting root input initializers: 1
2016-05-17 21:29:42,330 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:42,330 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D]
2016-05-17 21:29:42,333 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:42,333 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D]
2016-05-17 21:29:42,333 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D]
2016-05-17 21:29:42,333 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D]
2016-05-17 21:29:42,333 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=667DC6187DB14E41B2F1BC9A8A50F706, vertexId=vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:42,334 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=667DC6187DB14E41B2F1BC9A8A50F706, vertexId=vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D]
2016-05-17 21:29:42,335 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=667DC6187DB14E41B2F1BC9A8A50F706, vertexName=C9ADE6D058B34033AA904C47B080E93D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:42,336 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=667DC6187DB14E41B2F1BC9A8A50F706, vertexId=vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D]
2016-05-17 21:29:42,337 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:42,337 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C9ADE6D058B34033AA904C47B080E93D, vertexId=vertex_1463520582170_0001_1_01, initRequestedTime=1463520582330, initedTime=1463520582336, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:42,337 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:42,337 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 7E85923CAEF64820B344F0F42CC27621 on vertex vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011]
2016-05-17 21:29:42,347 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:42,347 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 7E85923CAEF64820B344F0F42CC27621 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:42,347 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:42,347 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:42,348 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:42,349 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 7E85923CAEF64820B344F0F42CC27621 on vertex vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011]
2016-05-17 21:29:42,349 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:42,349 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {7E85923CAEF64820B344F0F42CC27621=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:42,349 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011] parallelism set to 1
2016-05-17 21:29:42,349 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:42,349 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520582170_0001_1_00, eventCount=1
2016-05-17 21:29:42,350 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:42,350 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F06CAEFB6C3C44CCA2EF09508AE9D011, vertexId=vertex_1463520582170_0001_1_00, initRequestedTime=1463520582329, initedTime=1463520582349, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:42,350 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011] with distanceFromRoot: 0
2016-05-17 21:29:42,350 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:42,350 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F06CAEFB6C3C44CCA2EF09508AE9D011
2016-05-17 21:29:42,350 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011]
2016-05-17 21:29:42,350 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:42,351 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520582170_0001_1_00, startRequestedTime=1463520582350, startedTime=1463520582350
2016-05-17 21:29:42,351 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:42,351 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:42,351 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:TASK_STARTED]: vertexName=F06CAEFB6C3C44CCA2EF09508AE9D011, taskId=task_1463520582170_0001_1_00_000000, scheduledTime=1463520582351, launchTime=1463520582351
2016-05-17 21:29:42,351 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520582170_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:42,351 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520582170_0001_1_00 for vertex: vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D] numStartedSources: 1 numSources: 1
2016-05-17 21:29:42,351 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D] with distanceFromRoot: 1
2016-05-17 21:29:42,352 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011]
2016-05-17 21:29:42,352 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F06CAEFB6C3C44CCA2EF09508AE9D011 in vertex: C9ADE6D058B34033AA904C47B080E93D
2016-05-17 21:29:42,352 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C9ADE6D058B34033AA904C47B080E93D with 1 source tasks and 1 pending tasks
2016-05-17 21:29:42,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:42,352 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520582170_0001_1_01, startRequestedTime=1463520582352, startedTime=1463520582352
2016-05-17 21:29:42,352 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:42,353 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [CB722D543D444658AC54F32E99C5F08B/B1BDEEB2B54C4A62B95193242146BE8C] every/(1/1) ...mblyplatform/0_VALUES_ALL, VertexName: F06CAEFB6C3C44CCA2EF09508AE9D011, VertexParallelism: 1, TaskAttemptID:attempt_1463520582170_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7E85923CAEF64820B344F0F42CC27621, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C9ADE6D058B34033AA904C47B080E93D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:42,353 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520582170_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:42,353 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:42,354 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:42,354 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520582170_0001, with dag id: dag_1463520582170_0001_1
2016-05-17 21:29:42,354 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:42,354 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:42,354 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520582170_0001_00_000001
2016-05-17 21:29:42,355 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582170_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:42,356 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520582170_0001_00_000001
2016-05-17 21:29:42,356 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:42,357 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582170_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:42,357 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:42,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520582170_0001_00_000001, launchTime=1463520582357
2016-05-17 21:29:42,360 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520582170_0001_00_000001
2016-05-17 21:29:42,361 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520582170_0001_1_00_000000_0] to container: [container_1463520582170_0001_00_000001]
2016-05-17 21:29:42,362 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582170_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:42,362 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520582170_0001_00_000001 given task: attempt_1463520582170_0001_1_00_000000_0
2016-05-17 21:29:42,363 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520582170_0001_1_00_000000_0] started. Is using containerId: [container_1463520582170_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:42,363 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520582170_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520582170_0001_1_00_000000_0
2016-05-17 21:29:42,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F06CAEFB6C3C44CCA2EF09508AE9D011, taskAttemptId=attempt_1463520582170_0001_1_00_000000_0, startTime=1463520582362, containerId=container_1463520582170_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520582170_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:42,363 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:42 Starting to run new task attempt: attempt_1463520582170_0001_1_00_000000_0
2016-05-17 21:29:42,364 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520582170_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:42,364 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520582170_0001_1_00_000000_0
2016-05-17 21:29:42,364 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:42,364 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520582170_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:42,364 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:42,364 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:42,364 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:42,364 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:42,364 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:42,364 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CB722D543D444658AC54F32E99C5F08B/B1BDEEB2B54C4A62B95193242146BE8C] every/(1/1) ...mblyplatform/0_VALUES_ALL, VertexName: F06CAEFB6C3C44CCA2EF09508AE9D011, VertexParallelism: 1, TaskAttemptID:attempt_1463520582170_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7E85923CAEF64820B344F0F42CC27621, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C9ADE6D058B34033AA904C47B080E93D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:42,365 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:42,366 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520582170_0001_1_00_000000_0
2016-05-17 21:29:42,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:42,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:42,367 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520582170_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:42,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:42,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=7E85923CAEF64820B344F0F42CC27621, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:42,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:42,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 7E85923CAEF64820B344F0F42CC27621
2016-05-17 21:29:42,368 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 7E85923CAEF64820B344F0F42CC27621
2016-05-17 21:29:42,370 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:42,370 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:42,370 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 7E85923CAEF64820B344F0F42CC27621
2016-05-17 21:29:42,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 7E85923CAEF64820B344F0F42CC27621
2016-05-17 21:29:42,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C9ADE6D058B34033AA904C47B080E93D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:42,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:42,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C9ADE6D058B34033AA904C47B080E93D
2016-05-17 21:29:42,373 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:42,373 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C9ADE6D058B34033AA904C47B080E93D
2016-05-17 21:29:42,374 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C9ADE6D058B34033AA904C47B080E93D
2016-05-17 21:29:42,376 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F06CAEFB6C3C44CCA2EF09508AE9D011, ordinal: 0
2016-05-17 21:29:42,376 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F06CAEFB6C3C44CCA2EF09508AE9D011, mem on start (mb), free: 305, total: 362, max: 672
2016-05-17 21:29:42,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:42,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:42,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:42,377 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:42,377 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:42,378 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:42,378 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:42,378 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:42,379 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:42,379 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:42,379 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 7E85923CAEF64820B344F0F42CC27621, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:42,380 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C9ADE6D058B34033AA904C47B080E93D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:42,380 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:42,380 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 7E85923CAEF64820B344F0F42CC27621
2016-05-17 21:29:42,381 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:42,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 7E85923CAEF64820B344F0F42CC27621
2016-05-17 21:29:42,380 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 7E85923CAEF64820B344F0F42CC27621 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:42,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:42,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:42,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:42,382 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520582170_0001_1_00_000000_0
2016-05-17 21:29:42,384 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:42,384 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7f97c0e9
2016-05-17 21:29:42,386 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7f97c0e9
2016-05-17 21:29:42,386 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:42,386 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:42,387 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:42,387 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 7E85923CAEF64820B344F0F42CC27621
2016-05-17 21:29:42,387 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C7F34534EA19448A9479B5EFFDCC9783
2016-05-17 21:29:42,388 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C7F34534EA19448A9479B5EFFDCC9783
2016-05-17 21:29:42,389 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:42,392 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:42,392 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3a998062; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@33bbbffc; comparator=cascading.tuple.hadoop.util.TupleComparator@6b3953ec; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:42,393 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:42,393 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:42,393 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:42,406 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:42,406 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:42,407 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:42,407 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:42,407 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:42,407 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:42,407 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:42,408 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F06CAEFB6C3C44CCA2EF09508AE9D011, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:42,410 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520582170_0001_1_00_000000_0
2016-05-17 21:29:42,411 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F06CAEFB6C3C44CCA2EF09508AE9D011, mem on close (mb), free: 202, total: 362, max: 672
2016-05-17 21:29:42,411 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:42,411 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:42,411 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:42,411 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:42,415 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:42 Completed running task attempt: attempt_1463520582170_0001_1_00_000000_0
2016-05-17 21:29:42,418 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520582170_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:42,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379584512, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520582408, Process_Duration=2, Process_End_Time=1463520582410, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:42,418 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520582170_0001_1_00, eventCount=1
2016-05-17 21:29:42,418 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:42,418 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:42,418 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520582170_0001_00_000001
2016-05-17 21:29:42,418 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F06CAEFB6C3C44CCA2EF09508AE9D011, taskAttemptId=attempt_1463520582170_0001_1_00_000000_0, startTime=1463520582362, finishTime=1463520582418, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379584512, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520582408, Process_Duration=2, Process_End_Time=1463520582410, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:42,419 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520582170_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:42,419 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:42,419 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520582170_0001_1_00_000000_0
2016-05-17 21:29:42,419 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:42,419 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:42,419 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:TASK_FINISHED]: vertexName=F06CAEFB6C3C44CCA2EF09508AE9D011, taskId=task_1463520582170_0001_1_00_000000, startTime=1463520582362, finishTime=1463520582419, timeTaken=57, status=SUCCEEDED, successfulAttemptID=attempt_1463520582170_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379584512, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520582408, Process_Duration=2, Process_End_Time=1463520582410, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:42,420 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520582170_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:42,420 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011] : 1
2016-05-17 21:29:42,420 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011]
2016-05-17 21:29:42,420 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:42,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:VERTEX_FINISHED]: vertexName=F06CAEFB6C3C44CCA2EF09508AE9D011, vertexId=vertex_1463520582170_0001_1_00, initRequestedTime=1463520582329, initedTime=1463520582349, startRequestedTime=1463520582350, startedTime=1463520582350, finishTime=1463520582420, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379584512, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520582408, Process_Duration=2, Process_End_Time=1463520582410, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520582362, firstTasksToStart=[ task_1463520582170_0001_1_00_000000 ], lastTaskFinishTime=1463520582418, lastTasksToFinish=[ task_1463520582170_0001_1_00_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520582170_0001_1_00_000000 ], longestDurationTasks=[ task_1463520582170_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:42,421 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:42,421 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582170_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:42,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520582170_0001_00_000001, stoppedTime=1463520582421, exitStatus=0
2016-05-17 21:29:42,421 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:42,421 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582170_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:42,421 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D] attempt: attempt_1463520582170_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:42,422 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C9ADE6D058B34033AA904C47B080E93D
2016-05-17 21:29:42,422 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D]
2016-05-17 21:29:42,422 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:42,422 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520582170_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:42,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:TASK_STARTED]: vertexName=C9ADE6D058B34033AA904C47B080E93D, taskId=task_1463520582170_0001_1_01_000000, scheduledTime=1463520582422, launchTime=1463520582422
2016-05-17 21:29:42,422 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520582170_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:42,423 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582170_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:42,422 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:42,423 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D]
2016-05-17 21:29:42,423 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [CB722D543D444658AC54F32E99C5F08B/B1BDEEB2B54C4A62B95193242146BE8C] every/(1/1) ...mblyplatform/0_VALUES_ALL, VertexName: C9ADE6D058B34033AA904C47B080E93D, VertexParallelism: 1, TaskAttemptID:attempt_1463520582170_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F06CAEFB6C3C44CCA2EF09508AE9D011, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=667DC6187DB14E41B2F1BC9A8A50F706, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:42,423 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520582170_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:42,423 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:42,424 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520582170_0001_00_000002
2016-05-17 21:29:42,424 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582170_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:42,425 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520582170_0001_00_000002
2016-05-17 21:29:42,425 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:42,426 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582170_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:42,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:42,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520582170_0001_00_000002, launchTime=1463520582426
2016-05-17 21:29:42,427 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520582170_0001_00_000002
2016-05-17 21:29:42,427 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520582170_0001_1_01_000000_0] to container: [container_1463520582170_0001_00_000002]
2016-05-17 21:29:42,428 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582170_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:42,428 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520582170_0001_00_000002 given task: attempt_1463520582170_0001_1_01_000000_0
2016-05-17 21:29:42,429 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520582170_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:42,431 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520582170_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:42,428 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520582170_0001_1_01_000000_0] started. Is using containerId: [container_1463520582170_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:42,431 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520582170_0001_00_000001, Exiting
2016-05-17 21:29:42,432 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:42,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C9ADE6D058B34033AA904C47B080E93D, taskAttemptId=attempt_1463520582170_0001_1_01_000000_0, startTime=1463520582428, containerId=container_1463520582170_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520582170_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:42,432 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520582170_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:42,432 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520582170_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:42,431 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520582170_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520582170_0001_1_01_000000_0
2016-05-17 21:29:42,432 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520582170_0001_00_000001
2016-05-17 21:29:42,432 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520582170_0001_00_000001
2016-05-17 21:29:42 Starting to run new task attempt: attempt_1463520582170_0001_1_01_000000_0
2016-05-17 21:29:42,433 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520582170_0001_1_01_000000_0
2016-05-17 21:29:42,433 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:42,433 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:42,433 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:42,433 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:42,433 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:42,434 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:42,434 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [CB722D543D444658AC54F32E99C5F08B/B1BDEEB2B54C4A62B95193242146BE8C] every/(1/1) ...mblyplatform/0_VALUES_ALL, VertexName: C9ADE6D058B34033AA904C47B080E93D, VertexParallelism: 1, TaskAttemptID:attempt_1463520582170_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F06CAEFB6C3C44CCA2EF09508AE9D011, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=667DC6187DB14E41B2F1BC9A8A50F706, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:42,434 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:42,434 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520582170_0001_1_01_000000_0
2016-05-17 21:29:42,435 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:42,435 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:42,435 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:42,435 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520582170_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:42,435 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520582170_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:42,436 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520582170_0001_00_000001 completed successfully
2016-05-17 21:29:42,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F06CAEFB6C3C44CCA2EF09508AE9D011, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:42,436 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582170_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:42,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:42,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F06CAEFB6C3C44CCA2EF09508AE9D011
2016-05-17 21:29:42,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=667DC6187DB14E41B2F1BC9A8A50F706, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:42,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:42,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 667DC6187DB14E41B2F1BC9A8A50F706
2016-05-17 21:29:42,438 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:42,438 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:42,438 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:42,438 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F06CAEFB6C3C44CCA2EF09508AE9D011
2016-05-17 21:29:42,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F06CAEFB6C3C44CCA2EF09508AE9D011
2016-05-17 21:29:42,438 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:42,439 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 667DC6187DB14E41B2F1BC9A8A50F706
2016-05-17 21:29:42,443 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C9ADE6D058B34033AA904C47B080E93D, ordinal: 1
2016-05-17 21:29:42,443 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C9ADE6D058B34033AA904C47B080E93D, mem on start (mb), free: 197, total: 362, max: 672
2016-05-17 21:29:42,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:42,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:42,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:42,446 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:42,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 667DC6187DB14E41B2F1BC9A8A50F706
2016-05-17 21:29:42,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:42,447 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:42,447 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:42,447 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:42,447 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:42,448 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:42,448 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:42,448 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F06CAEFB6C3C44CCA2EF09508AE9D011, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:42,448 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 667DC6187DB14E41B2F1BC9A8A50F706, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:42,448 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:42,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F06CAEFB6C3C44CCA2EF09508AE9D011 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:42,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F06CAEFB6C3C44CCA2EF09508AE9D011
2016-05-17 21:29:42,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:42,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:42,449 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:42,450 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:42,450 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F06CAEFB6C3C44CCA2EF09508AE9D011 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:42,450 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:42,450 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:42,450 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:42,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F06CAEFB6C3C44CCA2EF09508AE9D011
2016-05-17 21:29:42,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:42,453 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520582170_0001_1_01_000000_0
2016-05-17 21:29:42,456 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:42,457 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C7F34534EA19448A9479B5EFFDCC9783
2016-05-17 21:29:42,457 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_ALL"], id: 667DC6187DB14E41B2F1BC9A8A50F706
2016-05-17 21:29:42,457 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_ALL"]
2016-05-17 21:29:42,457 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C7F34534EA19448A9479B5EFFDCC9783, for 1 inputs
2016-05-17 21:29:42,458 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520582170_0001_1_00_000000_0_10189, runDuration: 44924, ]
2016-05-17 21:29:42,459 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520582170_0001_1_00_000000_0_10189] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520582459, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:42,459 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F06CAEFB6C3C44CCA2EF09508AE9D011
2016-05-17 21:29:42,459 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:42,459 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [F06CAEFB6C3C44CCA2EF09508AE9D011] #32 in 1ms
2016-05-17 21:29:42,459 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F06CAEFB6C3C44CCA2EF09508AE9D011] #32, status:false, isInterrupted:false
2016-05-17 21:29:42,460 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:42,461 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:42,461 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:42,461 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:42,461 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:42,461 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:42,461 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F06CAEFB6C3C44CCA2EF09508AE9D011
2016-05-17 21:29:42,461 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:42,461 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C9ADE6D058B34033AA904C47B080E93D, all 1 inputs ready in: 00:00:00.004
2016-05-17 21:29:42,462 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:42,464 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@16d11a0e; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@7a0598d5
2016-05-17 21:29:42,465 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:42,465 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:42,465 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:42,465 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:42,466 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520582170_0001_1_01_000000_0
2016-05-17 21:29:42,466 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520582170_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:42,466 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520582170_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:42,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379584512, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520582461, Process_Duration=5, Process_End_Time=1463520582466, Read_Duration=1, Tuples_Read=1]]
2016-05-17 21:29:42,467 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C9ADE6D058B34033AA904C47B080E93D, mem on close (mb), free: 193, total: 362, max: 672
2016-05-17 21:29:42,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=C9ADE6D058B34033AA904C47B080E93D, index=1
2016-05-17 21:29:42,467 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F06CAEFB6C3C44CCA2EF09508AE9D011
2016-05-17 21:29:42,467 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C9ADE6D058B34033AA904C47B080E93D, taskAttemptId=attempt_1463520582170_0001_1_01_000000_0, startTime=1463520582428, finishTime=1463520582467, timeTaken=39, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379584512, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520582461, Process_Duration=5, Process_End_Time=1463520582466, Read_Duration=1, Tuples_Read=1
2016-05-17 21:29:42,470 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520582170_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:42,470 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:42,470 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520582170_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:42,470 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:TASK_FINISHED]: vertexName=C9ADE6D058B34033AA904C47B080E93D, taskId=task_1463520582170_0001_1_01_000000, startTime=1463520582428, finishTime=1463520582470, timeTaken=42, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:42,470 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520582170_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:42,470 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D] : 1
2016-05-17 21:29:42,470 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:42,470 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:42,470 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D] because task failed: task_1463520582170_0001_1_01_000000
2016-05-17 21:29:42,471 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:42,471 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:42,471 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D]
2016-05-17 21:29:42,471 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=667DC6187DB14E41B2F1BC9A8A50F706, vertexId=vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D]
2016-05-17 21:29:42,471 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:42,472 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:42,472 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:VERTEX_FINISHED]: vertexName=C9ADE6D058B34033AA904C47B080E93D, vertexId=vertex_1463520582170_0001_1_01, initRequestedTime=1463520582330, initedTime=1463520582336, startRequestedTime=1463520582352, startedTime=1463520582352, finishTime=1463520582472, timeTaken=120, status=FAILED, diagnostics=Task failed, taskId=task_1463520582170_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520582428, firstTasksToStart=[ task_1463520582170_0001_1_01_000000 ], lastTaskFinishTime=1463520582467, lastTasksToFinish=[ task_1463520582170_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:42,473 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:42,473 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520582170_0001_00_000002, stoppedTime=1463520582473, exitStatus=0
2016-05-17 21:29:42,473 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:42,473 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582170_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:42,473 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520582170_0001_1_01_000000_0 failed: true container: container_1463520582170_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:42,474 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:42,474 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:42,474 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:42,474 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:42,474 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:42,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=C9ADE6D058B34033AA904C47B080E93D, sourceVertex=F06CAEFB6C3C44CCA2EF09508AE9D011
2016-05-17 21:29:42,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=C9ADE6D058B34033AA904C47B080E93D, sourceVertex=667DC6187DB14E41B2F1BC9A8A50F706
2016-05-17 21:29:42,474 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:42,475 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:42,475 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520582170_0001_00_000002
2016-05-17 21:29:42,475 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520582170_0001_00_000002
2016-05-17 21:29:42,475 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520582170_0001_1
2016-05-17 21:29:42,475 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 667DC6187DB14E41B2F1BC9A8A50F706 for vertex: vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D]
2016-05-17 21:29:42,475 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520582170_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:42,475 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520582170_0001_00_000002 completed but with errors
2016-05-17 21:29:42,475 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520582170_0001_1_00 [F06CAEFB6C3C44CCA2EF09508AE9D011]
2016-05-17 21:29:42,476 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520582170_0001_1, queueSize=0
2016-05-17 21:29:42,476 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:42,476 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582170_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520582170_0001_1, startTime=1463520582328, finishTime=1463520582475, timeTaken=147, status=FAILED, diagnostics=Vertex failed, vertexName=C9ADE6D058B34033AA904C47B080E93D, vertexId=vertex_1463520582170_0001_1_01, diagnostics=[Task failed, taskId=task_1463520582170_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520582170_0001_1_01 [C9ADE6D058B34033AA904C47B080E93D] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=379584512, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520582408, Process_Duration=2, Process_End_Time=1463520582410, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:42,477 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520582170_0001_1 finished with state: FAILED
2016-05-17 21:29:42,477 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520582170_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:42,477 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520582170_0001_00_000002
2016-05-17 21:29:42,477 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520582170_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:42,478 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582170_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:42 Completed Dag: dag_1463520582170_0001_1
2016-05-17 21:29:42,478 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520582170_0001_1, dagState=FAILED
2016-05-17 21:29:42,478 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520582170_0001_1_post
2016-05-17 21:29:42,479 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:42,479 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:42,479 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520582170_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:42,481 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/0_VALUES_ALL, applicationId=application_1463520582170_0001
2016-05-17 21:29:42,482 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:42,482 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:42,482 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:42,483 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:42,483 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:42,483 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:42,483 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:42,483 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:42,483 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:42,483 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:42,484 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:42,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:42,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:42,488 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/CB722D543D444658AC54F32E99C5F08B/B1BDEEB2B54C4A62B95193242146BE8C/.tez/application_1463520582170_0001
2016-05-17 21:29:42,488 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:42,488 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...mblyplatform/0_VALUES_ALL
2016-05-17 21:29:42,489 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:42,489 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_ALL/_temporary
2016-05-17 21:29:42,489 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.335
2016-05-17 21:29:42,489 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_ALL
2016-05-17 21:29:42,511 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:42,511 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:42,542 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 21:29:42,543 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 21:29:42,543 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:42,544 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:42,544 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:42,564 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:42,570 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_REPLACE
2016-05-17 21:29:42,608 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:42,609 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:42,609 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_REPLACE"]
2016-05-17 21:29:42,609 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:42,609 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:42,610 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:42,610 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...platform/0_VALUES_REPLACE
2016-05-17 21:29:42,610 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:42,621 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:42,625 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:42,625 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:42,625 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:42,634 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5374F7B7DB424F2BB7634C235B4177EC/D45FA5DF5A27482B907376FFAB1CED7E/.tez/application_1463520582625_0001 doesn't exist and is created
2016-05-17 21:29:42,648 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:42,648 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:42,649 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5374F7B7DB424F2BB7634C235B4177EC/D45FA5DF5A27482B907376FFAB1CED7E/.tez/application_1463520582625_0001
2016-05-17 21:29:42,651 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520582625_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:42,662 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:42,662 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:42,663 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:42,663 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:42,664 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5374F7B7DB424F2BB7634C235B4177EC/D45FA5DF5A27482B907376FFAB1CED7E/.tez/application_1463520582625_0001/localmode-log-dir
2016-05-17 21:29:42,664 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5374F7B7DB424F2BB7634C235B4177EC/D45FA5DF5A27482B907376FFAB1CED7E/.tez/application_1463520582625_0001/localmode-log-dir/history.txt.appattempt_1463520582625_0001_000000, maxErrors=10
2016-05-17 21:29:42,664 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:42,664 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520582625_0001_000000, appSubmitTime=1463520582650, launchTime=1463520582650
2016-05-17 21:29:42,667 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:42,668 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:42,669 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:42,669 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:42,671 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:54340
2016-05-17 21:29:42,674 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:42,675 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520582625_0001_000000, startTime=1463520582675
2016-05-17 21:29:42,675 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:42,675 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:42,685 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:42,685 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:42,748 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:42,748 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:42,749 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_REPLACE, applicationId=application_1463520582625_0001, dagName=[5374F7B7DB424F2BB7634C235B4177EC/D45FA5DF5A27482B907376FFAB1CED7E] every/(1/1) ...platform/0_VALUES_REPLACE
2016-05-17 21:29:42,764 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5374F7B7DB424F2BB7634C235B4177EC/D45FA5DF5A27482B907376FFAB1CED7E] every/(1/1) ...platform/0_VALUES_REPLACE
2016-05-17 21:29:42,765 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520582625_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5374F7B7DB424F2BB7634C235B4177EC/D45FA5DF5A27482B907376FFAB1CED7E/.tez/application_1463520582625_0001/localmode-log-dir/dag_1463520582625_0001_1.dot
2016-05-17 21:29:42,766 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5374F7B7DB424F2BB7634C235B4177EC/D45FA5DF5A27482B907376FFAB1CED7E/.tez/application_1463520582625_0001/localmode-log-dir/dag_1463520582625_0001_1-tez-dag.pb.txt
2016-05-17 21:29:42,771 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520582625_0001_1
2016-05-17 21:29:42,771 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:42,771 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5374F7B7DB424F2BB7634C235B4177EC/D45FA5DF5A27482B907376FFAB1CED7E] every/(1/1) ...platform/0_VALUES_REPLACE
2016-05-17 21:29:42 Running Dag: dag_1463520582625_0001_1
2016-05-17 21:29:42,782 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:42,782 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520582625_0001_1, submitTime=1463520582764
2016-05-17 21:29:42,783 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D748A5A7BCAD48489C0DB6C2D882B1C0
2016-05-17 21:29:42,783 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:42,783 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:42,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520582625_0001_1, initTime=1463520582782
2016-05-17 21:29:42,784 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520582625_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:42,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:DAG_STARTED]: dagID=dag_1463520582625_0001_1, startTime=1463520582784
2016-05-17 21:29:42,784 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:42,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:42,784 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520582625_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:42,784 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EA14E301F8BF4A01A008D6697E11119D : {2AD09F3DE4584913889361A2594F5FE0={InputName=2AD09F3DE4584913889361A2594F5FE0}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:42,784 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2AD09F3DE4584913889361A2594F5FE0, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:42,784 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D]
2016-05-17 21:29:42,784 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D]
2016-05-17 21:29:42,785 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D]
2016-05-17 21:29:42,785 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D]. Starting root input initializers: 1
2016-05-17 21:29:42,785 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:42,785 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0]
2016-05-17 21:29:42,785 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2AD09F3DE4584913889361A2594F5FE0 on vertex vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D]
2016-05-17 21:29:42,787 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:42,787 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2AD09F3DE4584913889361A2594F5FE0 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:42,787 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:42,788 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:42,788 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_REPLACE, applicationId=application_1463520582625_0001, dagName=[5374F7B7DB424F2BB7634C235B4177EC/D45FA5DF5A27482B907376FFAB1CED7E] every/(1/1) ...platform/0_VALUES_REPLACE
2016-05-17 21:29:42,788 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:42,788 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0]
2016-05-17 21:29:42,788 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0]
2016-05-17 21:29:42,788 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0]
2016-05-17 21:29:42,789 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0113A1000B904260857515092D8E7A9D, vertexId=vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:42,789 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0113A1000B904260857515092D8E7A9D, vertexId=vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0]
2016-05-17 21:29:42,789 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:42,789 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2AD09F3DE4584913889361A2594F5FE0 on vertex vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D]
2016-05-17 21:29:42,790 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0113A1000B904260857515092D8E7A9D, vertexName=D748A5A7BCAD48489C0DB6C2D882B1C0, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:42,790 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0113A1000B904260857515092D8E7A9D, vertexId=vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0]
2016-05-17 21:29:42,791 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:42,791 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D748A5A7BCAD48489C0DB6C2D882B1C0, vertexId=vertex_1463520582625_0001_1_01, initRequestedTime=1463520582785, initedTime=1463520582791, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:42,791 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:42,792 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:42,792 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2AD09F3DE4584913889361A2594F5FE0=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:42,792 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D] parallelism set to 1
2016-05-17 21:29:42,792 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:42,793 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520582625_0001_1_00, eventCount=1
2016-05-17 21:29:42,793 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:42,793 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EA14E301F8BF4A01A008D6697E11119D, vertexId=vertex_1463520582625_0001_1_00, initRequestedTime=1463520582784, initedTime=1463520582793, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:42,793 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D] with distanceFromRoot: 0
2016-05-17 21:29:42,793 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:42,794 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EA14E301F8BF4A01A008D6697E11119D
2016-05-17 21:29:42,794 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D]
2016-05-17 21:29:42,794 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:42,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520582625_0001_1_00, startRequestedTime=1463520582794, startedTime=1463520582794
2016-05-17 21:29:42,794 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:42,795 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:42,795 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:TASK_STARTED]: vertexName=EA14E301F8BF4A01A008D6697E11119D, taskId=task_1463520582625_0001_1_00_000000, scheduledTime=1463520582795, launchTime=1463520582795
2016-05-17 21:29:42,795 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520582625_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:42,795 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520582625_0001_1_00 for vertex: vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0] numStartedSources: 1 numSources: 1
2016-05-17 21:29:42,796 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0] with distanceFromRoot: 1
2016-05-17 21:29:42,796 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D]
2016-05-17 21:29:42,796 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: EA14E301F8BF4A01A008D6697E11119D in vertex: D748A5A7BCAD48489C0DB6C2D882B1C0
2016-05-17 21:29:42,797 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D748A5A7BCAD48489C0DB6C2D882B1C0 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:42,797 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:42,797 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520582625_0001_1_01, startRequestedTime=1463520582796, startedTime=1463520582796
2016-05-17 21:29:42,797 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:42,798 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5374F7B7DB424F2BB7634C235B4177EC/D45FA5DF5A27482B907376FFAB1CED7E] every/(1/1) ...platform/0_VALUES_REPLACE, VertexName: EA14E301F8BF4A01A008D6697E11119D, VertexParallelism: 1, TaskAttemptID:attempt_1463520582625_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2AD09F3DE4584913889361A2594F5FE0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D748A5A7BCAD48489C0DB6C2D882B1C0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:42,798 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520582625_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:42,798 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:42,799 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:42,799 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:42,799 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:42,799 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520582625_0001_00_000001
2016-05-17 21:29:42,799 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582625_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:42,801 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520582625_0001_00_000001
2016-05-17 21:29:42,801 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:42,801 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582625_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:42,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520582625_0001_00_000001, launchTime=1463520582801
2016-05-17 21:29:42,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:42,802 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520582625_0001_00_000001
2016-05-17 21:29:42,803 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520582625_0001_1_00_000000_0] to container: [container_1463520582625_0001_00_000001]
2016-05-17 21:29:42,803 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582625_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:42,803 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520582625_0001_1_00_000000_0] started. Is using containerId: [container_1463520582625_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:42,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EA14E301F8BF4A01A008D6697E11119D, taskAttemptId=attempt_1463520582625_0001_1_00_000000_0, startTime=1463520582803, containerId=container_1463520582625_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520582625_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:42,803 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520582625_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:42,804 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520582625_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:42,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:42,804 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520582625_0001_00_000001 given task: attempt_1463520582625_0001_1_00_000000_0
2016-05-17 21:29:42,804 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520582625_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520582625_0001_1_00_000000_0
2016-05-17 21:29:42 Starting to run new task attempt: attempt_1463520582625_0001_1_00_000000_0
2016-05-17 21:29:42,804 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520582625_0001_1_00_000000_0
2016-05-17 21:29:42,805 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:42,805 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:42,805 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:42,805 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:42,805 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:42,805 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:42,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5374F7B7DB424F2BB7634C235B4177EC/D45FA5DF5A27482B907376FFAB1CED7E] every/(1/1) ...platform/0_VALUES_REPLACE, VertexName: EA14E301F8BF4A01A008D6697E11119D, VertexParallelism: 1, TaskAttemptID:attempt_1463520582625_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2AD09F3DE4584913889361A2594F5FE0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D748A5A7BCAD48489C0DB6C2D882B1C0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:42,805 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:42,807 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520582625_0001_1_00_000000_0
2016-05-17 21:29:42,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:42,807 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:42,808 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520582625_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:42,808 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:42,808 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D748A5A7BCAD48489C0DB6C2D882B1C0, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:42,808 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:42,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D748A5A7BCAD48489C0DB6C2D882B1C0
2016-05-17 21:29:42,810 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:42,810 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D748A5A7BCAD48489C0DB6C2D882B1C0
2016-05-17 21:29:42,810 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D748A5A7BCAD48489C0DB6C2D882B1C0
2016-05-17 21:29:42,810 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520582625_0001, with dag id: dag_1463520582625_0001_1
2016-05-17 21:29:42,811 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:42,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2AD09F3DE4584913889361A2594F5FE0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:42,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:42,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2AD09F3DE4584913889361A2594F5FE0
2016-05-17 21:29:42,815 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2AD09F3DE4584913889361A2594F5FE0
2016-05-17 21:29:42,815 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EA14E301F8BF4A01A008D6697E11119D, ordinal: 0
2016-05-17 21:29:42,815 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EA14E301F8BF4A01A008D6697E11119D, mem on start (mb), free: 118, total: 362, max: 672
2016-05-17 21:29:42,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:42,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:42,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:42,817 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:42,817 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2AD09F3DE4584913889361A2594F5FE0
2016-05-17 21:29:42,817 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2AD09F3DE4584913889361A2594F5FE0
2016-05-17 21:29:42,817 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:42,818 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:42,818 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:42,818 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:42,818 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:42,818 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:42,818 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:42,819 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D748A5A7BCAD48489C0DB6C2D882B1C0, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:42,819 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2AD09F3DE4584913889361A2594F5FE0, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:42,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:42,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2AD09F3DE4584913889361A2594F5FE0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:42,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:42,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:42,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2AD09F3DE4584913889361A2594F5FE0
2016-05-17 21:29:42,820 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:42,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2AD09F3DE4584913889361A2594F5FE0
2016-05-17 21:29:42,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:42,820 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520582625_0001_1_00_000000_0
2016-05-17 21:29:42,820 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:42,821 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@37d5e75e
2016-05-17 21:29:42,822 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@37d5e75e
2016-05-17 21:29:42,822 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:42,822 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:42,823 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:42,823 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2AD09F3DE4584913889361A2594F5FE0
2016-05-17 21:29:42,823 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: F0C585C718D54FD9B756A96B3566C475
2016-05-17 21:29:42,824 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] F0C585C718D54FD9B756A96B3566C475
2016-05-17 21:29:42,825 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:42,828 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:42,828 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3a9d9bc0; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@415c7850; comparator=cascading.tuple.hadoop.util.TupleComparator@5d59d69c; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:42,829 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:42,829 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:42,829 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:42,855 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:42,855 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:42,855 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:42,855 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:42,855 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:42,856 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:42,856 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:42,856 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EA14E301F8BF4A01A008D6697E11119D, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:42,859 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520582625_0001_1_00_000000_0
2016-05-17 21:29:42,859 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EA14E301F8BF4A01A008D6697E11119D, mem on close (mb), free: 210, total: 362, max: 672
2016-05-17 21:29:42,859 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:42,859 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:42,859 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:42,859 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:42,863 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:42 Completed running task attempt: attempt_1463520582625_0001_1_00_000000_0
2016-05-17 21:29:42,865 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520582625_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:42,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520582856, Process_Duration=3, Process_End_Time=1463520582859, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:42,866 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520582625_0001_1_00, eventCount=1
2016-05-17 21:29:42,866 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:42,866 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:42,866 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520582625_0001_00_000001
2016-05-17 21:29:42,866 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EA14E301F8BF4A01A008D6697E11119D, taskAttemptId=attempt_1463520582625_0001_1_00_000000_0, startTime=1463520582803, finishTime=1463520582866, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520582856, Process_Duration=3, Process_End_Time=1463520582859, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:42,866 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520582625_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:42,867 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:42,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:42,867 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520582625_0001_1_00_000000_0
2016-05-17 21:29:42,868 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:42,868 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:TASK_FINISHED]: vertexName=EA14E301F8BF4A01A008D6697E11119D, taskId=task_1463520582625_0001_1_00_000000, startTime=1463520582803, finishTime=1463520582867, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1463520582625_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520582856, Process_Duration=3, Process_End_Time=1463520582859, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:42,868 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520582625_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:42,868 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D] : 1
2016-05-17 21:29:42,868 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D]
2016-05-17 21:29:42,869 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:42,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:VERTEX_FINISHED]: vertexName=EA14E301F8BF4A01A008D6697E11119D, vertexId=vertex_1463520582625_0001_1_00, initRequestedTime=1463520582784, initedTime=1463520582793, startRequestedTime=1463520582794, startedTime=1463520582794, finishTime=1463520582868, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520582856, Process_Duration=3, Process_End_Time=1463520582859, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520582803, firstTasksToStart=[ task_1463520582625_0001_1_00_000000 ], lastTaskFinishTime=1463520582866, lastTasksToFinish=[ task_1463520582625_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520582625_0001_1_00_000000 ], longestDurationTasks=[ task_1463520582625_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:42,869 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:42,869 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582625_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:42,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520582625_0001_00_000001, stoppedTime=1463520582869, exitStatus=0
2016-05-17 21:29:42,869 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:42,869 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582625_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:42,870 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0] attempt: attempt_1463520582625_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:42,870 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D748A5A7BCAD48489C0DB6C2D882B1C0
2016-05-17 21:29:42,870 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0]
2016-05-17 21:29:42,870 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:42,870 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520582625_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:42,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:TASK_STARTED]: vertexName=D748A5A7BCAD48489C0DB6C2D882B1C0, taskId=task_1463520582625_0001_1_01_000000, scheduledTime=1463520582870, launchTime=1463520582870
2016-05-17 21:29:42,870 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:42,870 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520582625_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:42,871 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582625_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:42,871 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0]
2016-05-17 21:29:42,871 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5374F7B7DB424F2BB7634C235B4177EC/D45FA5DF5A27482B907376FFAB1CED7E] every/(1/1) ...platform/0_VALUES_REPLACE, VertexName: D748A5A7BCAD48489C0DB6C2D882B1C0, VertexParallelism: 1, TaskAttemptID:attempt_1463520582625_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EA14E301F8BF4A01A008D6697E11119D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0113A1000B904260857515092D8E7A9D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:42,871 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520582625_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:42,871 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:42,871 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520582625_0001_00_000002
2016-05-17 21:29:42,871 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582625_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:42,873 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520582625_0001_00_000002
2016-05-17 21:29:42,873 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:42,874 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582625_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:42,874 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:42,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520582625_0001_00_000002, launchTime=1463520582874
2016-05-17 21:29:42,875 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520582625_0001_00_000002
2016-05-17 21:29:42,875 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520582625_0001_1_01_000000_0] to container: [container_1463520582625_0001_00_000002]
2016-05-17 21:29:42,875 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582625_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:42,875 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520582625_0001_1_01_000000_0] started. Is using containerId: [container_1463520582625_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:42,876 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D748A5A7BCAD48489C0DB6C2D882B1C0, taskAttemptId=attempt_1463520582625_0001_1_01_000000_0, startTime=1463520582875, containerId=container_1463520582625_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520582625_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:42,876 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:42,876 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520582625_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:42,876 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520582625_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:42,876 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520582625_0001_00_000002 given task: attempt_1463520582625_0001_1_01_000000_0
2016-05-17 21:29:42,876 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520582625_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520582625_0001_1_01_000000_0
2016-05-17 21:29:42 Starting to run new task attempt: attempt_1463520582625_0001_1_01_000000_0
2016-05-17 21:29:42,876 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520582625_0001_1_01_000000_0
2016-05-17 21:29:42,876 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:42,877 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:42,877 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:42,877 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:42,877 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520582625_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:42,877 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520582625_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:42,877 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520582625_0001_00_000001, Exiting
2016-05-17 21:29:42,877 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520582625_0001_00_000001
2016-05-17 21:29:42,877 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520582625_0001_00_000001
2016-05-17 21:29:42,877 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:42,878 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520582625_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:42,878 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520582625_0001_00_000001 completed successfully
2016-05-17 21:29:42,878 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:42,878 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582625_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:42,878 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5374F7B7DB424F2BB7634C235B4177EC/D45FA5DF5A27482B907376FFAB1CED7E] every/(1/1) ...platform/0_VALUES_REPLACE, VertexName: D748A5A7BCAD48489C0DB6C2D882B1C0, VertexParallelism: 1, TaskAttemptID:attempt_1463520582625_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EA14E301F8BF4A01A008D6697E11119D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0113A1000B904260857515092D8E7A9D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:42,878 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:42,879 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520582625_0001_1_01_000000_0
2016-05-17 21:29:42,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:42,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:42,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EA14E301F8BF4A01A008D6697E11119D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:42,880 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520582625_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:42,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:42,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0113A1000B904260857515092D8E7A9D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:42,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:42,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EA14E301F8BF4A01A008D6697E11119D
2016-05-17 21:29:42,884 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:42,884 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:42,884 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: EA14E301F8BF4A01A008D6697E11119D
2016-05-17 21:29:42,884 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:42,884 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EA14E301F8BF4A01A008D6697E11119D
2016-05-17 21:29:42,886 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:42,886 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0113A1000B904260857515092D8E7A9D
2016-05-17 21:29:42,886 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:42,886 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0113A1000B904260857515092D8E7A9D
2016-05-17 21:29:42,889 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D748A5A7BCAD48489C0DB6C2D882B1C0, ordinal: 1
2016-05-17 21:29:42,889 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D748A5A7BCAD48489C0DB6C2D882B1C0, mem on start (mb), free: 204, total: 362, max: 672
2016-05-17 21:29:42,889 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:42,889 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:42,889 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:42,894 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:42,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0113A1000B904260857515092D8E7A9D
2016-05-17 21:29:42,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:42,895 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:42,895 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:42,895 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:42,895 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:42,895 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:42,896 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:42,896 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EA14E301F8BF4A01A008D6697E11119D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:42,896 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0113A1000B904260857515092D8E7A9D, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:42,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:42,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EA14E301F8BF4A01A008D6697E11119D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:42,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EA14E301F8BF4A01A008D6697E11119D
2016-05-17 21:29:42,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:42,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:42,897 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:42,897 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:42,897 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: EA14E301F8BF4A01A008D6697E11119D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:42,897 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:42,897 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:42,898 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:42,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EA14E301F8BF4A01A008D6697E11119D
2016-05-17 21:29:42,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:42,900 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520582625_0001_1_01_000000_0
2016-05-17 21:29:42,901 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520582625_0001_1_00_000000_0_10194, runDuration: 55676, ]
2016-05-17 21:29:42,902 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520582625_0001_1_00_000000_0_10194] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520582902, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:42,903 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:42,904 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: F0C585C718D54FD9B756A96B3566C475
2016-05-17 21:29:42,904 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_REPLACE"], id: 0113A1000B904260857515092D8E7A9D
2016-05-17 21:29:42,904 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_REPLACE"]
2016-05-17 21:29:42,904 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] F0C585C718D54FD9B756A96B3566C475, for 1 inputs
2016-05-17 21:29:42,904 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : EA14E301F8BF4A01A008D6697E11119D
2016-05-17 21:29:42,905 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:42,905 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [EA14E301F8BF4A01A008D6697E11119D] #33 in 4ms
2016-05-17 21:29:42,905 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [EA14E301F8BF4A01A008D6697E11119D] #33, status:false, isInterrupted:false
2016-05-17 21:29:42,905 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:42,906 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:42,906 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:42,906 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:42,906 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:42,907 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:42,907 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : EA14E301F8BF4A01A008D6697E11119D
2016-05-17 21:29:42,907 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:42,907 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D748A5A7BCAD48489C0DB6C2D882B1C0, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:42,908 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:42,909 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@4e481c53; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@603bb54b
2016-05-17 21:29:42,910 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:42,910 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:42,911 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:42,911 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:42,911 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520582625_0001_1_01_000000_0
2016-05-17 21:29:42,911 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520582625_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:42,912 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520582625_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:42,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520582907, Process_Duration=5, Process_End_Time=1463520582912, Read_Duration=0, Tuples_Read=1]]
2016-05-17 21:29:42,913 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D748A5A7BCAD48489C0DB6C2D882B1C0, mem on close (mb), free: 200, total: 362, max: 672
2016-05-17 21:29:42,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=D748A5A7BCAD48489C0DB6C2D882B1C0, index=1
2016-05-17 21:29:42,913 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D748A5A7BCAD48489C0DB6C2D882B1C0, taskAttemptId=attempt_1463520582625_0001_1_01_000000_0, startTime=1463520582875, finishTime=1463520582913, timeTaken=38, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520582907, Process_Duration=5, Process_End_Time=1463520582912, Read_Duration=0, Tuples_Read=1
2016-05-17 21:29:42,913 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520582625_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:42,913 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:42,913 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520582625_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:42,913 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: EA14E301F8BF4A01A008D6697E11119D
2016-05-17 21:29:42,914 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:TASK_FINISHED]: vertexName=D748A5A7BCAD48489C0DB6C2D882B1C0, taskId=task_1463520582625_0001_1_01_000000, startTime=1463520582875, finishTime=1463520582914, timeTaken=39, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:42,914 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520582625_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:42,914 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:42,914 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:42,914 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0] : 1
2016-05-17 21:29:42,915 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0] because task failed: task_1463520582625_0001_1_01_000000
2016-05-17 21:29:42,915 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:42,914 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:42,915 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:42,915 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0]
2016-05-17 21:29:42,915 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=0113A1000B904260857515092D8E7A9D, vertexId=vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0]
2016-05-17 21:29:42,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:VERTEX_FINISHED]: vertexName=D748A5A7BCAD48489C0DB6C2D882B1C0, vertexId=vertex_1463520582625_0001_1_01, initRequestedTime=1463520582785, initedTime=1463520582791, startRequestedTime=1463520582796, startedTime=1463520582796, finishTime=1463520582916, timeTaken=120, status=FAILED, diagnostics=Task failed, taskId=task_1463520582625_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520582875, firstTasksToStart=[ task_1463520582625_0001_1_01_000000 ], lastTaskFinishTime=1463520582913, lastTasksToFinish=[ task_1463520582625_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:42,917 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:42,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520582625_0001_00_000002, stoppedTime=1463520582917, exitStatus=0
2016-05-17 21:29:42,917 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582625_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:42,917 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520582625_0001_1_01_000000_0 failed: true container: container_1463520582625_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:42,917 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:42,917 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:42,917 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520582625_0001_1
2016-05-17 21:29:42,917 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 0113A1000B904260857515092D8E7A9D for vertex: vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0]
2016-05-17 21:29:42,918 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520582625_0001_1_00 [EA14E301F8BF4A01A008D6697E11119D]
2016-05-17 21:29:42,918 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:42,918 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:42,919 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520582625_0001_1, queueSize=0
2016-05-17 21:29:42,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520582625_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520582625_0001_1, startTime=1463520582784, finishTime=1463520582918, timeTaken=134, status=FAILED, diagnostics=Vertex failed, vertexName=D748A5A7BCAD48489C0DB6C2D882B1C0, vertexId=vertex_1463520582625_0001_1_01, diagnostics=[Task failed, taskId=task_1463520582625_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520582625_0001_1_01 [D748A5A7BCAD48489C0DB6C2D882B1C0] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520582856, Process_Duration=3, Process_End_Time=1463520582859, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:42,920 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:42,921 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:42,921 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:42,922 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:42,922 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:42,922 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:42,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=D748A5A7BCAD48489C0DB6C2D882B1C0, sourceVertex=EA14E301F8BF4A01A008D6697E11119D
2016-05-17 21:29:42,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=D748A5A7BCAD48489C0DB6C2D882B1C0, sourceVertex=0113A1000B904260857515092D8E7A9D
2016-05-17 21:29:42,922 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520582625_0001_00_000002
2016-05-17 21:29:42,922 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520582625_0001_00_000002
2016-05-17 21:29:42,920 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520582625_0001_1 finished with state: FAILED
2016-05-17 21:29:42,922 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520582625_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:42,923 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520582625_0001_00_000002 completed but with errors
2016-05-17 21:29:42,923 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520582625_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:42,923 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520582625_0001_00_000002
2016-05-17 21:29:42 Completed Dag: dag_1463520582625_0001_1
2016-05-17 21:29:42,924 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520582625_0001_1, dagState=FAILED
2016-05-17 21:29:42,924 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520582625_0001_1_post
2016-05-17 21:29:42,924 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:42,924 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:42,925 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520582625_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:42,925 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520582625_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:42,925 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520582625_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:42,928 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/0_VALUES_REPLACE, applicationId=application_1463520582625_0001
2016-05-17 21:29:42,930 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:42,930 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:42,930 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:42,930 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:42,930 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:42,931 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:42,931 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:42,931 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:42,931 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:42,931 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:42,932 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:42,932 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:42,932 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:42,937 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/5374F7B7DB424F2BB7634C235B4177EC/D45FA5DF5A27482B907376FFAB1CED7E/.tez/application_1463520582625_0001
2016-05-17 21:29:42,937 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:42,937 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...platform/0_VALUES_REPLACE
2016-05-17 21:29:42,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:42,938 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_REPLACE/_temporary
2016-05-17 21:29:42,938 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.330
2016-05-17 21:29:42,938 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_REPLACE
2016-05-17 21:29:42,964 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:42,964 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:42,997 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 21:29:43,004 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-17 21:29:43,004 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:43,005 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:43,005 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:43,020 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:43,024 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_SWAP
2016-05-17 21:29:43,063 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:43,063 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:43,063 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_SWAP"]
2016-05-17 21:29:43,063 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:43,063 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:43,063 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:43,065 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...blyplatform/0_VALUES_SWAP
2016-05-17 21:29:43,065 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:43,075 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:43,080 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:43,080 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:43,080 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:43,089 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EC81FF8A305847E6B1379B2D3D53FFCC/D6ADD9A4FD93408FA39DAB5D17A51E8F/.tez/application_1463520583080_0001 doesn't exist and is created
2016-05-17 21:29:43,104 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:43,104 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:43,104 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EC81FF8A305847E6B1379B2D3D53FFCC/D6ADD9A4FD93408FA39DAB5D17A51E8F/.tez/application_1463520583080_0001
2016-05-17 21:29:43,106 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520583080_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:43,115 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:43,116 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:43,118 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:43,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:43,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EC81FF8A305847E6B1379B2D3D53FFCC/D6ADD9A4FD93408FA39DAB5D17A51E8F/.tez/application_1463520583080_0001/localmode-log-dir
2016-05-17 21:29:43,119 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EC81FF8A305847E6B1379B2D3D53FFCC/D6ADD9A4FD93408FA39DAB5D17A51E8F/.tez/application_1463520583080_0001/localmode-log-dir/history.txt.appattempt_1463520583080_0001_000000, maxErrors=10
2016-05-17 21:29:43,119 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:43,120 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520583080_0001_000000, appSubmitTime=1463520583106, launchTime=1463520583106
2016-05-17 21:29:43,123 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34442
2016-05-17 21:29:43,128 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:43,129 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:43,129 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:43,129 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:43,130 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:43,131 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520583080_0001_000000, startTime=1463520583131
2016-05-17 21:29:43,131 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:43,131 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:43,204 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:43,204 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:43,204 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/0_VALUES_SWAP, applicationId=application_1463520583080_0001, dagName=[EC81FF8A305847E6B1379B2D3D53FFCC/D6ADD9A4FD93408FA39DAB5D17A51E8F] every/(1/1) ...blyplatform/0_VALUES_SWAP
2016-05-17 21:29:43,218 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [EC81FF8A305847E6B1379B2D3D53FFCC/D6ADD9A4FD93408FA39DAB5D17A51E8F] every/(1/1) ...blyplatform/0_VALUES_SWAP
2016-05-17 21:29:43,219 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520583080_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EC81FF8A305847E6B1379B2D3D53FFCC/D6ADD9A4FD93408FA39DAB5D17A51E8F/.tez/application_1463520583080_0001/localmode-log-dir/dag_1463520583080_0001_1.dot
2016-05-17 21:29:43,219 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EC81FF8A305847E6B1379B2D3D53FFCC/D6ADD9A4FD93408FA39DAB5D17A51E8F/.tez/application_1463520583080_0001/localmode-log-dir/dag_1463520583080_0001_1-tez-dag.pb.txt
2016-05-17 21:29:43,226 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520583080_0001_1
2016-05-17 21:29:43,226 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:43,226 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [EC81FF8A305847E6B1379B2D3D53FFCC/D6ADD9A4FD93408FA39DAB5D17A51E8F] every/(1/1) ...blyplatform/0_VALUES_SWAP
2016-05-17 21:29:43 Running Dag: dag_1463520583080_0001_1
2016-05-17 21:29:43,237 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:43,237 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520583080_0001_1, submitTime=1463520583218
2016-05-17 21:29:43,238 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D86884E2D34C4FE1BBC9BF9B8C7F81ED
2016-05-17 21:29:43,238 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:43,238 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:43,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520583080_0001_1, initTime=1463520583238
2016-05-17 21:29:43,239 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520583080_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:43,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:DAG_STARTED]: dagID=dag_1463520583080_0001_1, startTime=1463520583239
2016-05-17 21:29:43,239 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:43,239 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:43,239 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/0_VALUES_SWAP, applicationId=application_1463520583080_0001, dagName=[EC81FF8A305847E6B1379B2D3D53FFCC/D6ADD9A4FD93408FA39DAB5D17A51E8F] every/(1/1) ...blyplatform/0_VALUES_SWAP
2016-05-17 21:29:43,239 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520583080_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:43,240 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E850550933AC4F29BE763E21599E8036 : {51883A732723416F9D97BBF2F06F265B={InputName=51883A732723416F9D97BBF2F06F265B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:43,240 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 51883A732723416F9D97BBF2F06F265B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:43,240 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036]
2016-05-17 21:29:43,240 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036]
2016-05-17 21:29:43,241 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036]
2016-05-17 21:29:43,241 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036]. Starting root input initializers: 1
2016-05-17 21:29:43,241 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:43,241 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED]
2016-05-17 21:29:43,242 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 51883A732723416F9D97BBF2F06F265B on vertex vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036]
2016-05-17 21:29:43,244 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:43,244 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:43,244 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED]
2016-05-17 21:29:43,245 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED]
2016-05-17 21:29:43,245 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED]
2016-05-17 21:29:43,245 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=78C21BC8D3B346658CB2C3A1921BA674, vertexId=vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:43,245 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=78C21BC8D3B346658CB2C3A1921BA674, vertexId=vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED]
2016-05-17 21:29:43,246 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 51883A732723416F9D97BBF2F06F265B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:43,246 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:43,246 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:43,247 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=78C21BC8D3B346658CB2C3A1921BA674, vertexName=D86884E2D34C4FE1BBC9BF9B8C7F81ED, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:43,247 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=78C21BC8D3B346658CB2C3A1921BA674, vertexId=vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED]
2016-05-17 21:29:43,248 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:43,248 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:43,249 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 51883A732723416F9D97BBF2F06F265B on vertex vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036]
2016-05-17 21:29:43,249 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D86884E2D34C4FE1BBC9BF9B8C7F81ED, vertexId=vertex_1463520583080_0001_1_01, initRequestedTime=1463520583241, initedTime=1463520583248, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:43,249 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:43,249 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:43,249 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {51883A732723416F9D97BBF2F06F265B=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:43,250 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036] parallelism set to 1
2016-05-17 21:29:43,250 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:43,250 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520583080_0001_1_00, eventCount=1
2016-05-17 21:29:43,251 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:43,251 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E850550933AC4F29BE763E21599E8036, vertexId=vertex_1463520583080_0001_1_00, initRequestedTime=1463520583240, initedTime=1463520583251, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:43,251 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036] with distanceFromRoot: 0
2016-05-17 21:29:43,252 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:43,252 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E850550933AC4F29BE763E21599E8036
2016-05-17 21:29:43,252 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036]
2016-05-17 21:29:43,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:43,252 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520583080_0001_1_00, startRequestedTime=1463520583252, startedTime=1463520583252
2016-05-17 21:29:43,253 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:43,253 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:43,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:TASK_STARTED]: vertexName=E850550933AC4F29BE763E21599E8036, taskId=task_1463520583080_0001_1_00_000000, scheduledTime=1463520583253, launchTime=1463520583253
2016-05-17 21:29:43,253 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520583080_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:43,254 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520583080_0001_1_00 for vertex: vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED] numStartedSources: 1 numSources: 1
2016-05-17 21:29:43,254 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED] with distanceFromRoot: 1
2016-05-17 21:29:43,254 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036]
2016-05-17 21:29:43,254 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E850550933AC4F29BE763E21599E8036 in vertex: D86884E2D34C4FE1BBC9BF9B8C7F81ED
2016-05-17 21:29:43,254 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D86884E2D34C4FE1BBC9BF9B8C7F81ED with 1 source tasks and 1 pending tasks
2016-05-17 21:29:43,255 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:43,255 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520583080_0001_1_01, startRequestedTime=1463520583254, startedTime=1463520583254
2016-05-17 21:29:43,255 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:43,255 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [EC81FF8A305847E6B1379B2D3D53FFCC/D6ADD9A4FD93408FA39DAB5D17A51E8F] every/(1/1) ...blyplatform/0_VALUES_SWAP, VertexName: E850550933AC4F29BE763E21599E8036, VertexParallelism: 1, TaskAttemptID:attempt_1463520583080_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=51883A732723416F9D97BBF2F06F265B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D86884E2D34C4FE1BBC9BF9B8C7F81ED, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:43,255 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520583080_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:43,256 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:43,256 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520583080_0001, with dag id: dag_1463520583080_0001_1
2016-05-17 21:29:43,256 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:43,256 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:43,257 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:43,257 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520583080_0001_00_000001
2016-05-17 21:29:43,257 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583080_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:43,258 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520583080_0001_00_000001
2016-05-17 21:29:43,259 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:43,259 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583080_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:43,259 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:43,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520583080_0001_00_000001, launchTime=1463520583259
2016-05-17 21:29:43,260 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520583080_0001_00_000001
2016-05-17 21:29:43,261 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520583080_0001_1_00_000000_0] to container: [container_1463520583080_0001_00_000001]
2016-05-17 21:29:43,261 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583080_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:43,261 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520583080_0001_1_00_000000_0] started. Is using containerId: [container_1463520583080_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:43,261 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E850550933AC4F29BE763E21599E8036, taskAttemptId=attempt_1463520583080_0001_1_00_000000_0, startTime=1463520583261, containerId=container_1463520583080_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520583080_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:43,262 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520583080_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:43,262 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:43,262 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520583080_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:43,262 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520583080_0001_00_000001 given task: attempt_1463520583080_0001_1_00_000000_0
2016-05-17 21:29:43,262 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520583080_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520583080_0001_1_00_000000_0
2016-05-17 21:29:43 Starting to run new task attempt: attempt_1463520583080_0001_1_00_000000_0
2016-05-17 21:29:43,263 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520583080_0001_1_00_000000_0
2016-05-17 21:29:43,263 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:43,263 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:43,263 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:43,263 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:43,264 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:43,264 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:43,264 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EC81FF8A305847E6B1379B2D3D53FFCC/D6ADD9A4FD93408FA39DAB5D17A51E8F] every/(1/1) ...blyplatform/0_VALUES_SWAP, VertexName: E850550933AC4F29BE763E21599E8036, VertexParallelism: 1, TaskAttemptID:attempt_1463520583080_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=51883A732723416F9D97BBF2F06F265B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D86884E2D34C4FE1BBC9BF9B8C7F81ED, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:43,264 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:43,265 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520583080_0001_1_00_000000_0
2016-05-17 21:29:43,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:43,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:43,265 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520583080_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:43,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:43,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D86884E2D34C4FE1BBC9BF9B8C7F81ED, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:43,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:43,266 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D86884E2D34C4FE1BBC9BF9B8C7F81ED
2016-05-17 21:29:43,267 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:43,268 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:43,268 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D86884E2D34C4FE1BBC9BF9B8C7F81ED
2016-05-17 21:29:43,268 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D86884E2D34C4FE1BBC9BF9B8C7F81ED
2016-05-17 21:29:43,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=51883A732723416F9D97BBF2F06F265B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:43,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:43,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 51883A732723416F9D97BBF2F06F265B
2016-05-17 21:29:43,271 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 51883A732723416F9D97BBF2F06F265B
2016-05-17 21:29:43,272 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E850550933AC4F29BE763E21599E8036, ordinal: 0
2016-05-17 21:29:43,272 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E850550933AC4F29BE763E21599E8036, mem on start (mb), free: 126, total: 362, max: 672
2016-05-17 21:29:43,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:43,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:43,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:43,272 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:43,272 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 51883A732723416F9D97BBF2F06F265B
2016-05-17 21:29:43,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 51883A732723416F9D97BBF2F06F265B
2016-05-17 21:29:43,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:43,273 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:43,273 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:43,273 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:43,273 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:43,274 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:43,274 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:43,274 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D86884E2D34C4FE1BBC9BF9B8C7F81ED, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:43,275 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 51883A732723416F9D97BBF2F06F265B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:43,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:43,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 51883A732723416F9D97BBF2F06F265B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:43,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:43,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:43,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 51883A732723416F9D97BBF2F06F265B
2016-05-17 21:29:43,276 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:43,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 51883A732723416F9D97BBF2F06F265B
2016-05-17 21:29:43,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:43,277 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520583080_0001_1_00_000000_0
2016-05-17 21:29:43,277 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:43,277 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7a8c9914
2016-05-17 21:29:43,278 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7a8c9914
2016-05-17 21:29:43,278 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:43,278 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:43,280 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:43,280 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 51883A732723416F9D97BBF2F06F265B
2016-05-17 21:29:43,280 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: A00296A61C4F49559D62BB2C4A7D66CB
2016-05-17 21:29:43,280 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] A00296A61C4F49559D62BB2C4A7D66CB
2016-05-17 21:29:43,281 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:43,283 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:43,283 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3c220740; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@5e0a1997; comparator=cascading.tuple.hadoop.util.TupleComparator@691638bc; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:43,283 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:43,284 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:43,284 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:43,309 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:43,309 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:43,309 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:43,309 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:43,310 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:43,310 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:43,310 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:43,310 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E850550933AC4F29BE763E21599E8036, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:43,315 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520583080_0001_1_00_000000_0
2016-05-17 21:29:43,316 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E850550933AC4F29BE763E21599E8036, mem on close (mb), free: 210, total: 363, max: 672
2016-05-17 21:29:43,316 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:43,316 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:43,316 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:43,316 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:43,320 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:43 Completed running task attempt: attempt_1463520583080_0001_1_00_000000_0
2016-05-17 21:29:43,323 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520583080_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:43,323 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520583080_0001_1_00, eventCount=1
2016-05-17 21:29:43,323 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520583310, Process_Duration=5, Process_End_Time=1463520583315, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:43,324 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E850550933AC4F29BE763E21599E8036, taskAttemptId=attempt_1463520583080_0001_1_00_000000_0, startTime=1463520583261, finishTime=1463520583323, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520583310, Process_Duration=5, Process_End_Time=1463520583315, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:43,324 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:43,324 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520583080_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:43,324 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520583080_0001_1_00_000000_0
2016-05-17 21:29:43,325 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:43,325 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:TASK_FINISHED]: vertexName=E850550933AC4F29BE763E21599E8036, taskId=task_1463520583080_0001_1_00_000000, startTime=1463520583261, finishTime=1463520583324, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1463520583080_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520583310, Process_Duration=5, Process_End_Time=1463520583315, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:43,325 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520583080_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:43,325 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036] : 1
2016-05-17 21:29:43,325 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036]
2016-05-17 21:29:43,326 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:43,326 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:VERTEX_FINISHED]: vertexName=E850550933AC4F29BE763E21599E8036, vertexId=vertex_1463520583080_0001_1_00, initRequestedTime=1463520583240, initedTime=1463520583251, startRequestedTime=1463520583252, startedTime=1463520583252, finishTime=1463520583325, timeTaken=73, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520583310, Process_Duration=5, Process_End_Time=1463520583315, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520583261, firstTasksToStart=[ task_1463520583080_0001_1_00_000000 ], lastTaskFinishTime=1463520583323, lastTasksToFinish=[ task_1463520583080_0001_1_00_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520583080_0001_1_00_000000 ], longestDurationTasks=[ task_1463520583080_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:43,326 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:43,326 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:43,326 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED] attempt: attempt_1463520583080_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:43,326 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D86884E2D34C4FE1BBC9BF9B8C7F81ED
2016-05-17 21:29:43,326 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED]
2016-05-17 21:29:43,327 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583080_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:43,327 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:43,327 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:TASK_STARTED]: vertexName=D86884E2D34C4FE1BBC9BF9B8C7F81ED, taskId=task_1463520583080_0001_1_01_000000, scheduledTime=1463520583327, launchTime=1463520583327
2016-05-17 21:29:43,327 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520583080_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:43,327 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED]
2016-05-17 21:29:43,329 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [EC81FF8A305847E6B1379B2D3D53FFCC/D6ADD9A4FD93408FA39DAB5D17A51E8F] every/(1/1) ...blyplatform/0_VALUES_SWAP, VertexName: D86884E2D34C4FE1BBC9BF9B8C7F81ED, VertexParallelism: 1, TaskAttemptID:attempt_1463520583080_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E850550933AC4F29BE763E21599E8036, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=78C21BC8D3B346658CB2C3A1921BA674, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:43,329 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:43,329 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520583080_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:43,329 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520583080_0001_00_000001, stoppedTime=1463520583329, exitStatus=0
2016-05-17 21:29:43,329 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583080_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:43,329 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520583080_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:43,329 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583080_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:43,330 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:43,330 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:43,330 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520583080_0001_00_000001
2016-05-17 21:29:43,329 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:43,329 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:43,330 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520583080_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:43,330 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520583080_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:43,329 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:43,331 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:43,330 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520583080_0001_00_000002
2016-05-17 21:29:43,331 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583080_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:43,332 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520583080_0001_00_000001, Exiting
2016-05-17 21:29:43,332 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520583080_0001_00_000001
2016-05-17 21:29:43,332 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520583080_0001_00_000001
2016-05-17 21:29:43,332 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520583080_0001_00_000002
2016-05-17 21:29:43,333 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520583080_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:43,333 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520583080_0001_00_000001 completed successfully
2016-05-17 21:29:43,333 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:43,333 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583080_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:43,334 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583080_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:43,335 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:43,335 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520583080_0001_00_000002, launchTime=1463520583334
2016-05-17 21:29:43,335 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520583080_0001_00_000002
2016-05-17 21:29:43,335 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520583080_0001_1_01_000000_0] to container: [container_1463520583080_0001_00_000002]
2016-05-17 21:29:43,335 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583080_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:43,336 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520583080_0001_00_000002 given task: attempt_1463520583080_0001_1_01_000000_0
2016-05-17 21:29:43,336 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520583080_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520583080_0001_1_01_000000_0
2016-05-17 21:29:43,336 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520583080_0001_1_01_000000_0] started. Is using containerId: [container_1463520583080_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:43,336 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:43,336 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D86884E2D34C4FE1BBC9BF9B8C7F81ED, taskAttemptId=attempt_1463520583080_0001_1_01_000000_0, startTime=1463520583336, containerId=container_1463520583080_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520583080_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:43,336 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520583080_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:43,336 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520583080_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:43 Starting to run new task attempt: attempt_1463520583080_0001_1_01_000000_0
2016-05-17 21:29:43,337 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520583080_0001_1_01_000000_0
2016-05-17 21:29:43,337 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:43,337 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:43,337 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:43,337 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:43,337 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:43,337 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:43,337 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EC81FF8A305847E6B1379B2D3D53FFCC/D6ADD9A4FD93408FA39DAB5D17A51E8F] every/(1/1) ...blyplatform/0_VALUES_SWAP, VertexName: D86884E2D34C4FE1BBC9BF9B8C7F81ED, VertexParallelism: 1, TaskAttemptID:attempt_1463520583080_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E850550933AC4F29BE763E21599E8036, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=78C21BC8D3B346658CB2C3A1921BA674, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:43,337 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:43,338 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520583080_0001_1_01_000000_0
2016-05-17 21:29:43,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:43,338 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:43,338 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520583080_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:43,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E850550933AC4F29BE763E21599E8036, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:43,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:43,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:43,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E850550933AC4F29BE763E21599E8036
2016-05-17 21:29:43,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=78C21BC8D3B346658CB2C3A1921BA674, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:43,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:43,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 78C21BC8D3B346658CB2C3A1921BA674
2016-05-17 21:29:43,340 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:43,340 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 78C21BC8D3B346658CB2C3A1921BA674
2016-05-17 21:29:43,342 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:43,343 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:43,343 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:43,343 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E850550933AC4F29BE763E21599E8036
2016-05-17 21:29:43,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E850550933AC4F29BE763E21599E8036
2016-05-17 21:29:43,346 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D86884E2D34C4FE1BBC9BF9B8C7F81ED, ordinal: 1
2016-05-17 21:29:43,346 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D86884E2D34C4FE1BBC9BF9B8C7F81ED, mem on start (mb), free: 203, total: 363, max: 672
2016-05-17 21:29:43,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:43,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:43,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:43,349 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:43,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 78C21BC8D3B346658CB2C3A1921BA674
2016-05-17 21:29:43,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:43,349 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:43,349 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:43,349 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:43,349 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:43,349 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:43,350 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:43,350 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 78C21BC8D3B346658CB2C3A1921BA674, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:43,350 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E850550933AC4F29BE763E21599E8036, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:43,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:43,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E850550933AC4F29BE763E21599E8036
2016-05-17 21:29:43,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E850550933AC4F29BE763E21599E8036 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:43,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:43,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:43,351 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:43,351 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:43,352 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E850550933AC4F29BE763E21599E8036 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:43,352 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:43,352 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:43,352 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:43,354 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E850550933AC4F29BE763E21599E8036
2016-05-17 21:29:43,355 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:43,355 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520583080_0001_1_01_000000_0
2016-05-17 21:29:43,358 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520583080_0001_1_00_000000_0_10200, runDuration: 55189, ]
2016-05-17 21:29:43,358 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:43,359 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: A00296A61C4F49559D62BB2C4A7D66CB
2016-05-17 21:29:43,359 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_SWAP"], id: 78C21BC8D3B346658CB2C3A1921BA674
2016-05-17 21:29:43,359 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520583080_0001_1_00_000000_0_10200] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520583359, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:43,360 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E850550933AC4F29BE763E21599E8036
2016-05-17 21:29:43,359 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_SWAP"]
2016-05-17 21:29:43,360 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] A00296A61C4F49559D62BB2C4A7D66CB, for 1 inputs
2016-05-17 21:29:43,360 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:43,360 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [E850550933AC4F29BE763E21599E8036] #34 in 1ms
2016-05-17 21:29:43,360 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E850550933AC4F29BE763E21599E8036] #34, status:false, isInterrupted:false
2016-05-17 21:29:43,361 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:43,362 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:43,362 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:43,362 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:43,362 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:43,362 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:43,362 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E850550933AC4F29BE763E21599E8036
2016-05-17 21:29:43,362 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:43,362 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D86884E2D34C4FE1BBC9BF9B8C7F81ED, all 1 inputs ready in: 00:00:00.002
2016-05-17 21:29:43,364 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:43,366 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@1f653aff; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@2f044b73
2016-05-17 21:29:43,367 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:43,367 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:43,369 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:43,369 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:43,369 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520583080_0001_1_01_000000_0
2016-05-17 21:29:43,369 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520583080_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:43,370 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520583080_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:43,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520583363, Process_Duration=7, Process_End_Time=1463520583370, Read_Duration=1, Tuples_Read=1]]
2016-05-17 21:29:43,373 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D86884E2D34C4FE1BBC9BF9B8C7F81ED, mem on close (mb), free: 200, total: 363, max: 672
2016-05-17 21:29:43,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=D86884E2D34C4FE1BBC9BF9B8C7F81ED, index=1
2016-05-17 21:29:43,373 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:43,372 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D86884E2D34C4FE1BBC9BF9B8C7F81ED, taskAttemptId=attempt_1463520583080_0001_1_01_000000_0, startTime=1463520583336, finishTime=1463520583372, timeTaken=36, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520583363, Process_Duration=7, Process_End_Time=1463520583370, Read_Duration=1, Tuples_Read=1
2016-05-17 21:29:43,373 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520583080_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:43,373 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520583080_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:43,374 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:TASK_FINISHED]: vertexName=D86884E2D34C4FE1BBC9BF9B8C7F81ED, taskId=task_1463520583080_0001_1_01_000000, startTime=1463520583336, finishTime=1463520583374, timeTaken=38, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:43,374 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520583080_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:43,374 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:43,374 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED] : 1
2016-05-17 21:29:43,374 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:43,374 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E850550933AC4F29BE763E21599E8036
2016-05-17 21:29:43,374 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED] because task failed: task_1463520583080_0001_1_01_000000
2016-05-17 21:29:43,374 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:43,374 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:43,375 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:43,375 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED]
2016-05-17 21:29:43,375 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=78C21BC8D3B346658CB2C3A1921BA674, vertexId=vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED]
2016-05-17 21:29:43,376 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:43,376 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:43,376 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:43,376 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:43,376 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:43,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=D86884E2D34C4FE1BBC9BF9B8C7F81ED, sourceVertex=E850550933AC4F29BE763E21599E8036
2016-05-17 21:29:43,376 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:VERTEX_FINISHED]: vertexName=D86884E2D34C4FE1BBC9BF9B8C7F81ED, vertexId=vertex_1463520583080_0001_1_01, initRequestedTime=1463520583241, initedTime=1463520583248, startRequestedTime=1463520583254, startedTime=1463520583254, finishTime=1463520583376, timeTaken=122, status=FAILED, diagnostics=Task failed, taskId=task_1463520583080_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520583336, firstTasksToStart=[ task_1463520583080_0001_1_01_000000 ], lastTaskFinishTime=1463520583372, lastTasksToFinish=[ task_1463520583080_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:43,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=D86884E2D34C4FE1BBC9BF9B8C7F81ED, sourceVertex=78C21BC8D3B346658CB2C3A1921BA674
2016-05-17 21:29:43,376 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:43,376 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:43,377 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520583080_0001_00_000002, stoppedTime=1463520583377, exitStatus=0
2016-05-17 21:29:43,377 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583080_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:43,377 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:43,377 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520583080_0001_1_01_000000_0 failed: true container: container_1463520583080_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:43,377 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:43,377 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:43,377 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520583080_0001_00_000002
2016-05-17 21:29:43,377 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520583080_0001_00_000002
2016-05-17 21:29:43,377 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520583080_0001_1
2016-05-17 21:29:43,378 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520583080_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:43,378 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520583080_0001_00_000002 completed but with errors
2016-05-17 21:29:43,378 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 78C21BC8D3B346658CB2C3A1921BA674 for vertex: vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED]
2016-05-17 21:29:43,378 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520583080_0001_1_00 [E850550933AC4F29BE763E21599E8036]
2016-05-17 21:29:43,379 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520583080_0001_1, queueSize=0
2016-05-17 21:29:43,380 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:43,380 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583080_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520583080_0001_1, startTime=1463520583239, finishTime=1463520583378, timeTaken=139, status=FAILED, diagnostics=Vertex failed, vertexName=D86884E2D34C4FE1BBC9BF9B8C7F81ED, vertexId=vertex_1463520583080_0001_1_01, diagnostics=[Task failed, taskId=task_1463520583080_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520583080_0001_1_01 [D86884E2D34C4FE1BBC9BF9B8C7F81ED] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520583310, Process_Duration=5, Process_End_Time=1463520583315, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:43,380 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520583080_0001_1 finished with state: FAILED
2016-05-17 21:29:43,380 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520583080_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:43,380 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520583080_0001_00_000002
2016-05-17 21:29:43,380 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520583080_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:43,380 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583080_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:43 Completed Dag: dag_1463520583080_0001_1
2016-05-17 21:29:43,381 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520583080_0001_1, dagState=FAILED
2016-05-17 21:29:43,381 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520583080_0001_1_post
2016-05-17 21:29:43,381 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:43,381 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:43,381 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520583080_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:43,386 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/0_VALUES_SWAP, applicationId=application_1463520583080_0001
2016-05-17 21:29:43,387 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:43,387 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:43,387 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:43,387 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:43,387 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:43,388 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:43,388 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:43,388 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:43,388 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:43,388 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:43,388 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:43,389 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:43,389 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:43,392 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EC81FF8A305847E6B1379B2D3D53FFCC/D6ADD9A4FD93408FA39DAB5D17A51E8F/.tez/application_1463520583080_0001
2016-05-17 21:29:43,393 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:43,393 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...blyplatform/0_VALUES_SWAP
2016-05-17 21:29:43,393 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:43,393 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_SWAP/_temporary
2016-05-17 21:29:43,394 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.330
2016-05-17 21:29:43,394 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_SWAP
2016-05-17 21:29:43,418 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:43,419 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:43,447 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 21:29:43,452 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 21:29:43,453 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:43,453 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:43,453 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:43,465 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:43,470 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_-1
2016-05-17 21:29:43,510 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:43,510 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:43,510 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_-1"]
2016-05-17 21:29:43,510 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:43,510 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:43,510 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:43,511 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...ssemblyplatform/0_ARGS_-1
2016-05-17 21:29:43,511 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:43,522 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:43,526 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:43,526 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:43,526 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:43,534 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/142F9FABC4134349859464566DCE7F04/FB677878852E4C6ABF6FE18761956BB3/.tez/application_1463520583526_0001 doesn't exist and is created
2016-05-17 21:29:43,549 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:43,549 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:43,549 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/142F9FABC4134349859464566DCE7F04/FB677878852E4C6ABF6FE18761956BB3/.tez/application_1463520583526_0001
2016-05-17 21:29:43,551 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520583526_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:43,564 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:43,565 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:43,566 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:43,566 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:43,567 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/142F9FABC4134349859464566DCE7F04/FB677878852E4C6ABF6FE18761956BB3/.tez/application_1463520583526_0001/localmode-log-dir
2016-05-17 21:29:43,567 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/142F9FABC4134349859464566DCE7F04/FB677878852E4C6ABF6FE18761956BB3/.tez/application_1463520583526_0001/localmode-log-dir/history.txt.appattempt_1463520583526_0001_000000, maxErrors=10
2016-05-17 21:29:43,567 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:43,568 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520583526_0001_000000, appSubmitTime=1463520583551, launchTime=1463520583551
2016-05-17 21:29:43,571 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:43,572 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:43,574 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:43,574 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:43,576 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43396
2016-05-17 21:29:43,577 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:43,579 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520583526_0001_000000, startTime=1463520583579
2016-05-17 21:29:43,579 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:43,579 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:43,649 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:43,649 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:43,649 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ssemblyplatform/0_ARGS_-1, applicationId=application_1463520583526_0001, dagName=[142F9FABC4134349859464566DCE7F04/FB677878852E4C6ABF6FE18761956BB3] every/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-05-17 21:29:43,663 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [142F9FABC4134349859464566DCE7F04/FB677878852E4C6ABF6FE18761956BB3] every/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-05-17 21:29:43,665 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520583526_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/142F9FABC4134349859464566DCE7F04/FB677878852E4C6ABF6FE18761956BB3/.tez/application_1463520583526_0001/localmode-log-dir/dag_1463520583526_0001_1.dot
2016-05-17 21:29:43,665 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/142F9FABC4134349859464566DCE7F04/FB677878852E4C6ABF6FE18761956BB3/.tez/application_1463520583526_0001/localmode-log-dir/dag_1463520583526_0001_1-tez-dag.pb.txt
2016-05-17 21:29:43,671 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520583526_0001_1
2016-05-17 21:29:43,671 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:43,671 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [142F9FABC4134349859464566DCE7F04/FB677878852E4C6ABF6FE18761956BB3] every/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-05-17 21:29:43 Running Dag: dag_1463520583526_0001_1
2016-05-17 21:29:43,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520583526_0001_1, submitTime=1463520583664
2016-05-17 21:29:43,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:43,683 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F797E5C7D4E745EE979748DA4EAB8EC5
2016-05-17 21:29:43,684 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:43,684 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:43,684 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520583526_0001_1, initTime=1463520583683
2016-05-17 21:29:43,684 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520583526_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:43,684 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:DAG_STARTED]: dagID=dag_1463520583526_0001_1, startTime=1463520583684
2016-05-17 21:29:43,685 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:43,684 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:43,685 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ssemblyplatform/0_ARGS_-1, applicationId=application_1463520583526_0001, dagName=[142F9FABC4134349859464566DCE7F04/FB677878852E4C6ABF6FE18761956BB3] every/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-05-17 21:29:43,685 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520583526_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:43,685 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E0EF872CA12C4843B845F2BC4EAD5987 : {913A27CEF04F48F4939BA62307D4E463={InputName=913A27CEF04F48F4939BA62307D4E463}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:43,685 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 913A27CEF04F48F4939BA62307D4E463, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:43,685 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987]
2016-05-17 21:29:43,685 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987]
2016-05-17 21:29:43,685 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987]
2016-05-17 21:29:43,686 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987]. Starting root input initializers: 1
2016-05-17 21:29:43,686 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:43,686 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5]
2016-05-17 21:29:43,687 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 913A27CEF04F48F4939BA62307D4E463 on vertex vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987]
2016-05-17 21:29:43,688 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:43,688 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 913A27CEF04F48F4939BA62307D4E463 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:43,689 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:43,689 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:43,689 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5]
2016-05-17 21:29:43,690 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:43,690 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5]
2016-05-17 21:29:43,690 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5]
2016-05-17 21:29:43,690 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=27F9A65D1F4249E78595282D2E01BA7E, vertexId=vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:43,691 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=27F9A65D1F4249E78595282D2E01BA7E, vertexId=vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5]
2016-05-17 21:29:43,691 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:43,692 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 913A27CEF04F48F4939BA62307D4E463 on vertex vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987]
2016-05-17 21:29:43,693 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=27F9A65D1F4249E78595282D2E01BA7E, vertexName=F797E5C7D4E745EE979748DA4EAB8EC5, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:43,693 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=27F9A65D1F4249E78595282D2E01BA7E, vertexId=vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5]
2016-05-17 21:29:43,694 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:43,694 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F797E5C7D4E745EE979748DA4EAB8EC5, vertexId=vertex_1463520583526_0001_1_01, initRequestedTime=1463520583686, initedTime=1463520583694, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:43,694 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:43,695 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:43,695 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {913A27CEF04F48F4939BA62307D4E463=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:43,695 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987] parallelism set to 1
2016-05-17 21:29:43,695 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:43,696 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520583526_0001_1_00, eventCount=1
2016-05-17 21:29:43,696 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:43,696 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E0EF872CA12C4843B845F2BC4EAD5987, vertexId=vertex_1463520583526_0001_1_00, initRequestedTime=1463520583685, initedTime=1463520583696, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:43,697 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987] with distanceFromRoot: 0
2016-05-17 21:29:43,697 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:43,697 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E0EF872CA12C4843B845F2BC4EAD5987
2016-05-17 21:29:43,697 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987]
2016-05-17 21:29:43,698 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:43,698 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520583526_0001_1_00, startRequestedTime=1463520583697, startedTime=1463520583697
2016-05-17 21:29:43,698 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:43,699 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:43,699 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:TASK_STARTED]: vertexName=E0EF872CA12C4843B845F2BC4EAD5987, taskId=task_1463520583526_0001_1_00_000000, scheduledTime=1463520583699, launchTime=1463520583699
2016-05-17 21:29:43,699 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520583526_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:43,699 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520583526_0001_1_00 for vertex: vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5] numStartedSources: 1 numSources: 1
2016-05-17 21:29:43,700 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5] with distanceFromRoot: 1
2016-05-17 21:29:43,700 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987]
2016-05-17 21:29:43,700 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E0EF872CA12C4843B845F2BC4EAD5987 in vertex: F797E5C7D4E745EE979748DA4EAB8EC5
2016-05-17 21:29:43,701 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F797E5C7D4E745EE979748DA4EAB8EC5 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:43,701 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:43,701 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520583526_0001_1_01, startRequestedTime=1463520583700, startedTime=1463520583700
2016-05-17 21:29:43,701 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:43,702 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [142F9FABC4134349859464566DCE7F04/FB677878852E4C6ABF6FE18761956BB3] every/(1/1) ...ssemblyplatform/0_ARGS_-1, VertexName: E0EF872CA12C4843B845F2BC4EAD5987, VertexParallelism: 1, TaskAttemptID:attempt_1463520583526_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=913A27CEF04F48F4939BA62307D4E463, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F797E5C7D4E745EE979748DA4EAB8EC5, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:43,702 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520583526_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:43,702 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:43,703 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:43,703 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520583526_0001, with dag id: dag_1463520583526_0001_1
2016-05-17 21:29:43,703 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:43,703 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:43,704 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520583526_0001_00_000001
2016-05-17 21:29:43,704 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583526_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:43,706 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520583526_0001_00_000001
2016-05-17 21:29:43,706 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:43,707 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:43,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520583526_0001_00_000001, launchTime=1463520583707
2016-05-17 21:29:43,707 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583526_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:43,708 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520583526_0001_00_000001
2016-05-17 21:29:43,708 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520583526_0001_1_00_000000_0] to container: [container_1463520583526_0001_00_000001]
2016-05-17 21:29:43,708 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583526_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:43,709 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520583526_0001_00_000001 given task: attempt_1463520583526_0001_1_00_000000_0
2016-05-17 21:29:43,709 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520583526_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520583526_0001_1_00_000000_0
2016-05-17 21:29:43,709 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520583526_0001_1_00_000000_0] started. Is using containerId: [container_1463520583526_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:43 Starting to run new task attempt: attempt_1463520583526_0001_1_00_000000_0
2016-05-17 21:29:43,709 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:43,709 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520583526_0001_1_00_000000_0
2016-05-17 21:29:43,709 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E0EF872CA12C4843B845F2BC4EAD5987, taskAttemptId=attempt_1463520583526_0001_1_00_000000_0, startTime=1463520583709, containerId=container_1463520583526_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520583526_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:43,710 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520583526_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:43,710 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:43,710 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520583526_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:43,710 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:43,710 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:43,710 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:43,710 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:43,711 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:43,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [142F9FABC4134349859464566DCE7F04/FB677878852E4C6ABF6FE18761956BB3] every/(1/1) ...ssemblyplatform/0_ARGS_-1, VertexName: E0EF872CA12C4843B845F2BC4EAD5987, VertexParallelism: 1, TaskAttemptID:attempt_1463520583526_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=913A27CEF04F48F4939BA62307D4E463, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F797E5C7D4E745EE979748DA4EAB8EC5, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:43,711 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:43,711 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520583526_0001_1_00_000000_0
2016-05-17 21:29:43,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:43,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:43,712 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520583526_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:43,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:43,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F797E5C7D4E745EE979748DA4EAB8EC5, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:43,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=913A27CEF04F48F4939BA62307D4E463, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:43,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:43,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F797E5C7D4E745EE979748DA4EAB8EC5
2016-05-17 21:29:43,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:43,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 913A27CEF04F48F4939BA62307D4E463
2016-05-17 21:29:43,714 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 913A27CEF04F48F4939BA62307D4E463
2016-05-17 21:29:43,714 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:43,715 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F797E5C7D4E745EE979748DA4EAB8EC5
2016-05-17 21:29:43,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F797E5C7D4E745EE979748DA4EAB8EC5
2016-05-17 21:29:43,715 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:43,716 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:43,716 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 913A27CEF04F48F4939BA62307D4E463
2016-05-17 21:29:43,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 913A27CEF04F48F4939BA62307D4E463
2016-05-17 21:29:43,719 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E0EF872CA12C4843B845F2BC4EAD5987, ordinal: 0
2016-05-17 21:29:43,719 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0EF872CA12C4843B845F2BC4EAD5987, mem on start (mb), free: 124, total: 363, max: 672
2016-05-17 21:29:43,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:43,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:43,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:43,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:43,720 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:43,720 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:43,720 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:43,720 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:43,721 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:43,721 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:43,721 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 913A27CEF04F48F4939BA62307D4E463, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:43,721 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F797E5C7D4E745EE979748DA4EAB8EC5, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:43,721 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:43,721 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 913A27CEF04F48F4939BA62307D4E463 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:43,721 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:43,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:43,721 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 913A27CEF04F48F4939BA62307D4E463
2016-05-17 21:29:43,722 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:43,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 913A27CEF04F48F4939BA62307D4E463
2016-05-17 21:29:43,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:43,722 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520583526_0001_1_00_000000_0
2016-05-17 21:29:43,723 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:43,723 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@137d26f0
2016-05-17 21:29:43,724 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@137d26f0
2016-05-17 21:29:43,724 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:43,724 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:43,727 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:43,727 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 913A27CEF04F48F4939BA62307D4E463
2016-05-17 21:29:43,728 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 9EB8F5B9317B40E3BEC1512D71597A80
2016-05-17 21:29:43,728 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 9EB8F5B9317B40E3BEC1512D71597A80
2016-05-17 21:29:43,730 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:43,735 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:43,736 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@140da031; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@a8ffb40; comparator=cascading.tuple.hadoop.util.TupleComparator@72b82c2a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:43,736 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:43,736 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:43,736 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:43,759 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:43,759 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:43,759 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:43,759 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:43,760 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:43,760 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:43,760 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:43,760 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E0EF872CA12C4843B845F2BC4EAD5987, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:43,763 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520583526_0001_1_00_000000_0
2016-05-17 21:29:43,763 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0EF872CA12C4843B845F2BC4EAD5987, mem on close (mb), free: 209, total: 363, max: 672
2016-05-17 21:29:43,763 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:43,763 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:43,763 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:43,763 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:43,766 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:43 Completed running task attempt: attempt_1463520583526_0001_1_00_000000_0
2016-05-17 21:29:43,768 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520583526_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:43,768 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520583761, Process_Duration=2, Process_End_Time=1463520583763, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:43,768 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520583526_0001_1_00, eventCount=1
2016-05-17 21:29:43,769 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E0EF872CA12C4843B845F2BC4EAD5987, taskAttemptId=attempt_1463520583526_0001_1_00_000000_0, startTime=1463520583709, finishTime=1463520583769, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520583761, Process_Duration=2, Process_End_Time=1463520583763, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:43,769 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520583526_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:43,769 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:43,769 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:43,769 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520583526_0001_1_00_000000_0
2016-05-17 21:29:43,770 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:43,770 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:43,770 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:TASK_FINISHED]: vertexName=E0EF872CA12C4843B845F2BC4EAD5987, taskId=task_1463520583526_0001_1_00_000000, startTime=1463520583709, finishTime=1463520583770, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1463520583526_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520583761, Process_Duration=2, Process_End_Time=1463520583763, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:43,771 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520583526_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:43,771 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987] : 1
2016-05-17 21:29:43,771 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987]
2016-05-17 21:29:43,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:VERTEX_FINISHED]: vertexName=E0EF872CA12C4843B845F2BC4EAD5987, vertexId=vertex_1463520583526_0001_1_00, initRequestedTime=1463520583685, initedTime=1463520583696, startRequestedTime=1463520583697, startedTime=1463520583697, finishTime=1463520583771, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520583761, Process_Duration=2, Process_End_Time=1463520583763, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520583709, firstTasksToStart=[ task_1463520583526_0001_1_00_000000 ], lastTaskFinishTime=1463520583769, lastTasksToFinish=[ task_1463520583526_0001_1_00_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520583526_0001_1_00_000000 ], longestDurationTasks=[ task_1463520583526_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:43,772 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:43,772 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583526_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:43,772 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5] attempt: attempt_1463520583526_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:43,772 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F797E5C7D4E745EE979748DA4EAB8EC5
2016-05-17 21:29:43,772 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5]
2016-05-17 21:29:43,772 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520583526_0001_00_000001, stoppedTime=1463520583772, exitStatus=0
2016-05-17 21:29:43,772 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583526_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:43,772 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:43,773 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:TASK_STARTED]: vertexName=F797E5C7D4E745EE979748DA4EAB8EC5, taskId=task_1463520583526_0001_1_01_000000, scheduledTime=1463520583773, launchTime=1463520583773
2016-05-17 21:29:43,773 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520583526_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:43,773 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5]
2016-05-17 21:29:43,773 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [142F9FABC4134349859464566DCE7F04/FB677878852E4C6ABF6FE18761956BB3] every/(1/1) ...ssemblyplatform/0_ARGS_-1, VertexName: F797E5C7D4E745EE979748DA4EAB8EC5, VertexParallelism: 1, TaskAttemptID:attempt_1463520583526_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E0EF872CA12C4843B845F2BC4EAD5987, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=27F9A65D1F4249E78595282D2E01BA7E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:43,770 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520583526_0001_00_000001
2016-05-17 21:29:43,773 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520583526_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:43,773 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520583526_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:43,773 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520583526_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:43,773 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520583526_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:43,771 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:43,774 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:43,774 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520583526_0001_00_000001, Exiting
2016-05-17 21:29:43,774 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520583526_0001_00_000001
2016-05-17 21:29:43,774 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520583526_0001_00_000001
2016-05-17 21:29:43,774 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:43,775 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:43,775 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:43,774 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583526_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:43,775 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520583526_0001_00_000002
2016-05-17 21:29:43,775 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583526_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:43,777 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520583526_0001_00_000002
2016-05-17 21:29:43,778 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:43,781 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520583526_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:43,781 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520583526_0001_00_000001 completed successfully
2016-05-17 21:29:43,782 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583526_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:43,782 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583526_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:43,782 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:43,782 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520583526_0001_00_000002, launchTime=1463520583782
2016-05-17 21:29:43,783 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520583526_0001_00_000002
2016-05-17 21:29:43,783 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520583526_0001_1_01_000000_0] to container: [container_1463520583526_0001_00_000002]
2016-05-17 21:29:43,784 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583526_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:43,784 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520583526_0001_1_01_000000_0] started. Is using containerId: [container_1463520583526_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:43,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F797E5C7D4E745EE979748DA4EAB8EC5, taskAttemptId=attempt_1463520583526_0001_1_01_000000_0, startTime=1463520583784, containerId=container_1463520583526_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520583526_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:43,784 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520583526_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:43,784 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520583526_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:43,785 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:43,785 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520583526_0001_00_000002 given task: attempt_1463520583526_0001_1_01_000000_0
2016-05-17 21:29:43,785 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520583526_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520583526_0001_1_01_000000_0
2016-05-17 21:29:43 Starting to run new task attempt: attempt_1463520583526_0001_1_01_000000_0
2016-05-17 21:29:43,785 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520583526_0001_1_01_000000_0
2016-05-17 21:29:43,785 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:43,785 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:43,785 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:43,786 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:43,786 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:43,786 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:43,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [142F9FABC4134349859464566DCE7F04/FB677878852E4C6ABF6FE18761956BB3] every/(1/1) ...ssemblyplatform/0_ARGS_-1, VertexName: F797E5C7D4E745EE979748DA4EAB8EC5, VertexParallelism: 1, TaskAttemptID:attempt_1463520583526_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E0EF872CA12C4843B845F2BC4EAD5987, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=27F9A65D1F4249E78595282D2E01BA7E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:43,786 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:43,787 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520583526_0001_1_01_000000_0
2016-05-17 21:29:43,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:43,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:43,788 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520583526_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:43,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:43,791 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:43,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E0EF872CA12C4843B845F2BC4EAD5987, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:43,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:43,793 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E0EF872CA12C4843B845F2BC4EAD5987
2016-05-17 21:29:43,795 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:43,796 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:43,796 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E0EF872CA12C4843B845F2BC4EAD5987
2016-05-17 21:29:43,796 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E0EF872CA12C4843B845F2BC4EAD5987
2016-05-17 21:29:43,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=27F9A65D1F4249E78595282D2E01BA7E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:43,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:43,798 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 27F9A65D1F4249E78595282D2E01BA7E
2016-05-17 21:29:43,798 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F797E5C7D4E745EE979748DA4EAB8EC5, ordinal: 1
2016-05-17 21:29:43,798 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F797E5C7D4E745EE979748DA4EAB8EC5, mem on start (mb), free: 203, total: 363, max: 672
2016-05-17 21:29:43,798 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:43,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:43,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:43,798 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:43,799 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 27F9A65D1F4249E78595282D2E01BA7E
2016-05-17 21:29:43,808 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:43,808 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 27F9A65D1F4249E78595282D2E01BA7E
2016-05-17 21:29:43,808 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:43,808 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:43,808 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:43,809 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:43,809 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:43,809 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:43,809 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:43,809 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E0EF872CA12C4843B845F2BC4EAD5987, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:43,809 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 27F9A65D1F4249E78595282D2E01BA7E, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:43,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:43,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E0EF872CA12C4843B845F2BC4EAD5987 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:43,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E0EF872CA12C4843B845F2BC4EAD5987
2016-05-17 21:29:43,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:43,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:43,810 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:43,811 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:43,811 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E0EF872CA12C4843B845F2BC4EAD5987 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:43,811 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:43,811 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:43,811 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:43,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E0EF872CA12C4843B845F2BC4EAD5987
2016-05-17 21:29:43,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:43,813 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520583526_0001_1_01_000000_0
2016-05-17 21:29:43,814 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520583526_0001_1_00_000000_0_10206, runDuration: 54183, ]
2016-05-17 21:29:43,815 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520583526_0001_1_00_000000_0_10206] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520583815, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:43,815 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:43,816 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E0EF872CA12C4843B845F2BC4EAD5987
2016-05-17 21:29:43,816 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:43,816 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [E0EF872CA12C4843B845F2BC4EAD5987] #35 in 1ms
2016-05-17 21:29:43,816 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 9EB8F5B9317B40E3BEC1512D71597A80
2016-05-17 21:29:43,816 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E0EF872CA12C4843B845F2BC4EAD5987] #35, status:false, isInterrupted:false
2016-05-17 21:29:43,816 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_-1"], id: 27F9A65D1F4249E78595282D2E01BA7E
2016-05-17 21:29:43,817 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:43,817 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_-1"]
2016-05-17 21:29:43,817 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 9EB8F5B9317B40E3BEC1512D71597A80, for 1 inputs
2016-05-17 21:29:43,819 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:43,819 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:43,819 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:43,819 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:43,819 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:43,819 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E0EF872CA12C4843B845F2BC4EAD5987
2016-05-17 21:29:43,819 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:43,820 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F797E5C7D4E745EE979748DA4EAB8EC5, all 1 inputs ready in: 00:00:00.002
2016-05-17 21:29:43,821 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:43,822 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@10e2d91b; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@1d1b8446
2016-05-17 21:29:43,825 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:43,825 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:43,826 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520583526_0001_1_01_000000_0
2016-05-17 21:29:43,826 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520583526_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:43,826 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520583526_0001_1_01_000000_0
2016-05-17 21:29:43,826 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F797E5C7D4E745EE979748DA4EAB8EC5, mem on close (mb), free: 200, total: 363, max: 672
2016-05-17 21:29:43,827 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E0EF872CA12C4843B845F2BC4EAD5987
2016-05-17 21:29:43,827 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:43,828 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:43,828 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:43,828 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:43,828 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:43,828 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:43 Completed running task attempt: attempt_1463520583526_0001_1_01_000000_0
2016-05-17 21:29:43,830 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520583526_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:43,830 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F797E5C7D4E745EE979748DA4EAB8EC5, taskAttemptId=attempt_1463520583526_0001_1_01_000000_0, startTime=1463520583784, finishTime=1463520583830, timeTaken=46, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520583820, Process_Duration=6, Process_End_Time=1463520583826, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:43,830 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:43,830 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520583526_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:43,830 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520583526_0001_1_01_000000_0
2016-05-17 21:29:43,831 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:43,831 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:TASK_FINISHED]: vertexName=F797E5C7D4E745EE979748DA4EAB8EC5, taskId=task_1463520583526_0001_1_01_000000, startTime=1463520583784, finishTime=1463520583830, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1463520583526_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520583820, Process_Duration=6, Process_End_Time=1463520583826, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:43,831 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520583526_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:43,831 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5] : 1
2016-05-17 21:29:43,831 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5]
2016-05-17 21:29:43,831 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:43,831 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:43,832 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:VERTEX_FINISHED]: vertexName=F797E5C7D4E745EE979748DA4EAB8EC5, vertexId=vertex_1463520583526_0001_1_01, initRequestedTime=1463520583686, initedTime=1463520583694, startRequestedTime=1463520583700, startedTime=1463520583700, finishTime=1463520583831, timeTaken=131, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520583820, Process_Duration=6, Process_End_Time=1463520583826, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520583784, firstTasksToStart=[ task_1463520583526_0001_1_01_000000 ], lastTaskFinishTime=1463520583830, lastTasksToFinish=[ task_1463520583526_0001_1_01_000000 ], minTaskDuration=46, maxTaskDuration=46, avgTaskDuration=46.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520583526_0001_1_01_000000 ], longestDurationTasks=[ task_1463520583526_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:43,832 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:43,832 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583526_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:43,832 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520583526_0001_00_000002, stoppedTime=1463520583832, exitStatus=0
2016-05-17 21:29:43,832 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:43,832 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583526_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:43,833 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:43,833 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520583526_0001_1
2016-05-17 21:29:43,833 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520583526_0001_1
2016-05-17 21:29:43,833 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520583526_0001_1_00 [E0EF872CA12C4843B845F2BC4EAD5987]
2016-05-17 21:29:43,833 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 27F9A65D1F4249E78595282D2E01BA7E for vertex: vertex_1463520583526_0001_1_01 [F797E5C7D4E745EE979748DA4EAB8EC5]
2016-05-17 21:29:43,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520583820, Process_Duration=6, Process_End_Time=1463520583826, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:43,834 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:43,835 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:43,835 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520583526_0001_00_000002
2016-05-17 21:29:43,835 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520583526_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:43,835 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520583526_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:43,836 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520583526_0001_00_000002, Exiting
2016-05-17 21:29:43,836 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520583526_0001_00_000002
2016-05-17 21:29:43,836 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520583526_0001_00_000002
2016-05-17 21:29:43,836 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520583526_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:43,836 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520583526_0001_00_000002 completed successfully
2016-05-17 21:29:43,843 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520583526_0001_1, queueSize=0
2016-05-17 21:29:43,843 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:43,843 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520583526_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520583526_0001_1, startTime=1463520583684, finishTime=1463520583840, timeTaken=156, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=305, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762314752, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041167581, Process_Duration=8, Process_End_Time=2927041167589, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:43,843 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520583526_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:43,843 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520583526_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:43,843 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520583526_0001_00_000002
2016-05-17 21:29:43,844 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520583526_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:43 Completed Dag: dag_1463520583526_0001_1
2016-05-17 21:29:43,845 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520583526_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:43,845 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520583526_0001_1_post
2016-05-17 21:29:43,846 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:43,846 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:43,849 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ssemblyplatform/0_ARGS_-1, applicationId=application_1463520583526_0001
2016-05-17 21:29:43,850 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:43,850 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:43,850 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:43,850 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:43,850 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:43,851 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:43,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:43,851 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:43,851 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:43,852 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:43,853 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:43,854 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:43,854 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:43,857 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/142F9FABC4134349859464566DCE7F04/FB677878852E4C6ABF6FE18761956BB3/.tez/application_1463520583526_0001
2016-05-17 21:29:43,858 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_-1/_temporary
2016-05-17 21:29:43,858 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.348
2016-05-17 21:29:43,865 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:43,865 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:43,919 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_-1
2016-05-17 21:29:43,938 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:43,938 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:43,961 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 21:29:43,963 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-17 21:29:43,964 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:43,964 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:43,964 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_'field'
2016-05-17 21:29:43,982 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:43,982 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:44,009 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 21:29:44,015 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 21:29:44,015 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:44,016 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:44,016 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:44,028 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:44,032 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_RESULTS
2016-05-17 21:29:44,071 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:44,071 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:44,071 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_RESULTS"]
2016-05-17 21:29:44,072 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:44,072 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:44,072 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:44,073 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...lyplatform/0_ARGS_RESULTS
2016-05-17 21:29:44,074 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:44,085 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:44,090 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:44,090 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:44,090 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:44,099 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A39DACA3B8C2403EAE500A93F3ECE853/CEFB43705F0A4AC98B68A3373E935621/.tez/application_1463520584090_0001 doesn't exist and is created
2016-05-17 21:29:44,114 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:44,115 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:44,115 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A39DACA3B8C2403EAE500A93F3ECE853/CEFB43705F0A4AC98B68A3373E935621/.tez/application_1463520584090_0001
2016-05-17 21:29:44,117 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520584090_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:44,126 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:44,127 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:44,128 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:44,128 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:44,128 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A39DACA3B8C2403EAE500A93F3ECE853/CEFB43705F0A4AC98B68A3373E935621/.tez/application_1463520584090_0001/localmode-log-dir
2016-05-17 21:29:44,129 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A39DACA3B8C2403EAE500A93F3ECE853/CEFB43705F0A4AC98B68A3373E935621/.tez/application_1463520584090_0001/localmode-log-dir/history.txt.appattempt_1463520584090_0001_000000, maxErrors=10
2016-05-17 21:29:44,129 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:44,129 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520584090_0001_000000, appSubmitTime=1463520584117, launchTime=1463520584117
2016-05-17 21:29:44,133 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34431
2016-05-17 21:29:44,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:44,137 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:44,138 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:44,138 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:44,139 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:44,140 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520584090_0001_000000, startTime=1463520584140
2016-05-17 21:29:44,140 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:44,140 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:44,215 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:44,215 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:44,215 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_RESULTS, applicationId=application_1463520584090_0001, dagName=[A39DACA3B8C2403EAE500A93F3ECE853/CEFB43705F0A4AC98B68A3373E935621] every/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-05-17 21:29:44,229 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A39DACA3B8C2403EAE500A93F3ECE853/CEFB43705F0A4AC98B68A3373E935621] every/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-05-17 21:29:44,230 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520584090_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A39DACA3B8C2403EAE500A93F3ECE853/CEFB43705F0A4AC98B68A3373E935621/.tez/application_1463520584090_0001/localmode-log-dir/dag_1463520584090_0001_1.dot
2016-05-17 21:29:44,231 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A39DACA3B8C2403EAE500A93F3ECE853/CEFB43705F0A4AC98B68A3373E935621/.tez/application_1463520584090_0001/localmode-log-dir/dag_1463520584090_0001_1-tez-dag.pb.txt
2016-05-17 21:29:44,236 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520584090_0001_1
2016-05-17 21:29:44,236 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:44,236 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A39DACA3B8C2403EAE500A93F3ECE853/CEFB43705F0A4AC98B68A3373E935621] every/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-05-17 21:29:44 Running Dag: dag_1463520584090_0001_1
2016-05-17 21:29:44,246 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:44,246 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520584090_0001_1, submitTime=1463520584229
2016-05-17 21:29:44,247 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D1EBB0E187294BC29207099B15BA961E
2016-05-17 21:29:44,247 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:44,247 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:44,247 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520584090_0001_1, initTime=1463520584246
2016-05-17 21:29:44,247 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520584090_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:44,248 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:DAG_STARTED]: dagID=dag_1463520584090_0001_1, startTime=1463520584248
2016-05-17 21:29:44,248 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:44,248 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520584090_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:44,248 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A5141C8B035547F2BE4D843B2C7D550C : {5E9EDCEEC72A41F98458F0D6CAB2A4C5={InputName=5E9EDCEEC72A41F98458F0D6CAB2A4C5}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:44,248 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 5E9EDCEEC72A41F98458F0D6CAB2A4C5, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:44,248 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C]
2016-05-17 21:29:44,248 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C]
2016-05-17 21:29:44,249 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C]
2016-05-17 21:29:44,249 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C]. Starting root input initializers: 1
2016-05-17 21:29:44,248 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:44,248 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_RESULTS, applicationId=application_1463520584090_0001, dagName=[A39DACA3B8C2403EAE500A93F3ECE853/CEFB43705F0A4AC98B68A3373E935621] every/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-05-17 21:29:44,249 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:44,249 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E]
2016-05-17 21:29:44,249 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 5E9EDCEEC72A41F98458F0D6CAB2A4C5 on vertex vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C]
2016-05-17 21:29:44,251 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:44,251 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 5E9EDCEEC72A41F98458F0D6CAB2A4C5 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:44,251 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:44,251 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:44,252 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:44,252 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E]
2016-05-17 21:29:44,252 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E]
2016-05-17 21:29:44,252 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E]
2016-05-17 21:29:44,252 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=863B14D615D84A389641AF4A49132894, vertexId=vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:44,253 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:44,253 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 5E9EDCEEC72A41F98458F0D6CAB2A4C5 on vertex vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C]
2016-05-17 21:29:44,253 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=863B14D615D84A389641AF4A49132894, vertexId=vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E]
2016-05-17 21:29:44,254 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=863B14D615D84A389641AF4A49132894, vertexName=D1EBB0E187294BC29207099B15BA961E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:44,254 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=863B14D615D84A389641AF4A49132894, vertexId=vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E]
2016-05-17 21:29:44,255 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:44,255 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D1EBB0E187294BC29207099B15BA961E, vertexId=vertex_1463520584090_0001_1_01, initRequestedTime=1463520584249, initedTime=1463520584255, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:44,255 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:44,256 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:44,256 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {5E9EDCEEC72A41F98458F0D6CAB2A4C5=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:44,256 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C] parallelism set to 1
2016-05-17 21:29:44,256 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:44,257 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520584090_0001_1_00, eventCount=1
2016-05-17 21:29:44,257 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:44,257 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A5141C8B035547F2BE4D843B2C7D550C, vertexId=vertex_1463520584090_0001_1_00, initRequestedTime=1463520584248, initedTime=1463520584257, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:44,257 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C] with distanceFromRoot: 0
2016-05-17 21:29:44,257 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:44,258 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A5141C8B035547F2BE4D843B2C7D550C
2016-05-17 21:29:44,258 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C]
2016-05-17 21:29:44,258 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:44,258 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520584090_0001_1_00, startRequestedTime=1463520584258, startedTime=1463520584258
2016-05-17 21:29:44,259 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:44,259 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:44,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:TASK_STARTED]: vertexName=A5141C8B035547F2BE4D843B2C7D550C, taskId=task_1463520584090_0001_1_00_000000, scheduledTime=1463520584259, launchTime=1463520584259
2016-05-17 21:29:44,260 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520584090_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:44,260 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520584090_0001_1_00 for vertex: vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E] numStartedSources: 1 numSources: 1
2016-05-17 21:29:44,260 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E] with distanceFromRoot: 1
2016-05-17 21:29:44,260 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C]
2016-05-17 21:29:44,261 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A5141C8B035547F2BE4D843B2C7D550C in vertex: D1EBB0E187294BC29207099B15BA961E
2016-05-17 21:29:44,261 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D1EBB0E187294BC29207099B15BA961E with 1 source tasks and 1 pending tasks
2016-05-17 21:29:44,261 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:44,261 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520584090_0001_1_01, startRequestedTime=1463520584260, startedTime=1463520584260
2016-05-17 21:29:44,262 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:44,262 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A39DACA3B8C2403EAE500A93F3ECE853/CEFB43705F0A4AC98B68A3373E935621] every/(1/1) ...lyplatform/0_ARGS_RESULTS, VertexName: A5141C8B035547F2BE4D843B2C7D550C, VertexParallelism: 1, TaskAttemptID:attempt_1463520584090_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5E9EDCEEC72A41F98458F0D6CAB2A4C5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D1EBB0E187294BC29207099B15BA961E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:44,262 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520584090_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:44,262 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:44,263 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:44,263 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:44,264 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:44,264 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520584090_0001, with dag id: dag_1463520584090_0001_1
2016-05-17 21:29:44,264 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520584090_0001_00_000001
2016-05-17 21:29:44,264 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584090_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:44,265 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520584090_0001_00_000001
2016-05-17 21:29:44,265 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:44,266 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584090_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:44,266 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:44,267 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520584090_0001_00_000001, launchTime=1463520584266
2016-05-17 21:29:44,267 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520584090_0001_00_000001
2016-05-17 21:29:44,268 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520584090_0001_1_00_000000_0] to container: [container_1463520584090_0001_00_000001]
2016-05-17 21:29:44,268 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584090_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:44,269 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520584090_0001_1_00_000000_0] started. Is using containerId: [container_1463520584090_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:44,269 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A5141C8B035547F2BE4D843B2C7D550C, taskAttemptId=attempt_1463520584090_0001_1_00_000000_0, startTime=1463520584268, containerId=container_1463520584090_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520584090_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:44,269 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520584090_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:44,269 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:44,269 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520584090_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:44,269 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520584090_0001_00_000001 given task: attempt_1463520584090_0001_1_00_000000_0
2016-05-17 21:29:44,269 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520584090_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520584090_0001_1_00_000000_0
2016-05-17 21:29:44 Starting to run new task attempt: attempt_1463520584090_0001_1_00_000000_0
2016-05-17 21:29:44,270 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520584090_0001_1_00_000000_0
2016-05-17 21:29:44,270 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:44,270 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:44,270 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:44,270 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:44,270 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:44,270 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:44,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A39DACA3B8C2403EAE500A93F3ECE853/CEFB43705F0A4AC98B68A3373E935621] every/(1/1) ...lyplatform/0_ARGS_RESULTS, VertexName: A5141C8B035547F2BE4D843B2C7D550C, VertexParallelism: 1, TaskAttemptID:attempt_1463520584090_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5E9EDCEEC72A41F98458F0D6CAB2A4C5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D1EBB0E187294BC29207099B15BA961E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:44,271 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:44,271 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520584090_0001_1_00_000000_0
2016-05-17 21:29:44,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:44,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:44,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=5E9EDCEEC72A41F98458F0D6CAB2A4C5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:44,272 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520584090_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:44,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D1EBB0E187294BC29207099B15BA961E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:44,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:44,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:44,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:44,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 5E9EDCEEC72A41F98458F0D6CAB2A4C5
2016-05-17 21:29:44,273 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 5E9EDCEEC72A41F98458F0D6CAB2A4C5
2016-05-17 21:29:44,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D1EBB0E187294BC29207099B15BA961E
2016-05-17 21:29:44,275 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:44,275 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 5E9EDCEEC72A41F98458F0D6CAB2A4C5
2016-05-17 21:29:44,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 5E9EDCEEC72A41F98458F0D6CAB2A4C5
2016-05-17 21:29:44,275 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:44,275 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:44,275 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D1EBB0E187294BC29207099B15BA961E
2016-05-17 21:29:44,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D1EBB0E187294BC29207099B15BA961E
2016-05-17 21:29:44,279 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A5141C8B035547F2BE4D843B2C7D550C, ordinal: 0
2016-05-17 21:29:44,279 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A5141C8B035547F2BE4D843B2C7D550C, mem on start (mb), free: 93, total: 363, max: 672
2016-05-17 21:29:44,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:44,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:44,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:44,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:44,298 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:44,298 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:44,298 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:44,298 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:44,298 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:44,298 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:44,298 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 5E9EDCEEC72A41F98458F0D6CAB2A4C5, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:44,298 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D1EBB0E187294BC29207099B15BA961E, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:44,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:44,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 5E9EDCEEC72A41F98458F0D6CAB2A4C5 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:44,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:44,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:44,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 5E9EDCEEC72A41F98458F0D6CAB2A4C5
2016-05-17 21:29:44,299 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:44,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 5E9EDCEEC72A41F98458F0D6CAB2A4C5
2016-05-17 21:29:44,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:44,300 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520584090_0001_1_00_000000_0
2016-05-17 21:29:44,300 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:44,300 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7228da38
2016-05-17 21:29:44,303 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:44,303 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7228da38
2016-05-17 21:29:44,303 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:44,303 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:44,303 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 5E9EDCEEC72A41F98458F0D6CAB2A4C5
2016-05-17 21:29:44,304 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 2FF692FC294F4FED83F74E49795349A2
2016-05-17 21:29:44,304 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 2FF692FC294F4FED83F74E49795349A2
2016-05-17 21:29:44,305 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:44,307 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:44,307 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@5921727f; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@22e41d90; comparator=cascading.tuple.hadoop.util.TupleComparator@2e50ead5; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:44,307 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:44,307 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:44,307 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:44,320 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:44,320 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:44,320 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:44,320 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:44,320 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:44,321 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:44,321 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:44,321 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A5141C8B035547F2BE4D843B2C7D550C, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:44,324 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520584090_0001_1_00_000000_0
2016-05-17 21:29:44,324 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A5141C8B035547F2BE4D843B2C7D550C, mem on close (mb), free: 204, total: 362, max: 672
2016-05-17 21:29:44,324 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:44,324 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:44,324 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:44,324 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:44,329 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:44 Completed running task attempt: attempt_1463520584090_0001_1_00_000000_0
2016-05-17 21:29:44,332 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520584090_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:44,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520584090_0001_1_00, eventCount=1
2016-05-17 21:29:44,332 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=17, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520584322, Process_Duration=2, Process_End_Time=1463520584324, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:44,333 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:44,333 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:44,333 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520584090_0001_00_000001
2016-05-17 21:29:44,333 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A5141C8B035547F2BE4D843B2C7D550C, taskAttemptId=attempt_1463520584090_0001_1_00_000000_0, startTime=1463520584268, finishTime=1463520584333, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=17, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520584322, Process_Duration=2, Process_End_Time=1463520584324, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:44,333 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:44,333 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520584090_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:44,333 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520584090_0001_1_00_000000_0
2016-05-17 21:29:44,334 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:44,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:TASK_FINISHED]: vertexName=A5141C8B035547F2BE4D843B2C7D550C, taskId=task_1463520584090_0001_1_00_000000, startTime=1463520584268, finishTime=1463520584333, timeTaken=65, status=SUCCEEDED, successfulAttemptID=attempt_1463520584090_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=17, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520584322, Process_Duration=2, Process_End_Time=1463520584324, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:44,334 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520584090_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:44,334 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C] : 1
2016-05-17 21:29:44,334 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C]
2016-05-17 21:29:44,335 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:44,334 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:44,335 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:VERTEX_FINISHED]: vertexName=A5141C8B035547F2BE4D843B2C7D550C, vertexId=vertex_1463520584090_0001_1_00, initRequestedTime=1463520584248, initedTime=1463520584257, startRequestedTime=1463520584258, startedTime=1463520584258, finishTime=1463520584334, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=17, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520584322, Process_Duration=2, Process_End_Time=1463520584324, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520584268, firstTasksToStart=[ task_1463520584090_0001_1_00_000000 ], lastTaskFinishTime=1463520584333, lastTasksToFinish=[ task_1463520584090_0001_1_00_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520584090_0001_1_00_000000 ], longestDurationTasks=[ task_1463520584090_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:44,335 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:44,335 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E] attempt: attempt_1463520584090_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:44,335 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D1EBB0E187294BC29207099B15BA961E
2016-05-17 21:29:44,335 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E]
2016-05-17 21:29:44,336 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584090_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:44,336 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:44,336 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:TASK_STARTED]: vertexName=D1EBB0E187294BC29207099B15BA961E, taskId=task_1463520584090_0001_1_01_000000, scheduledTime=1463520584336, launchTime=1463520584336
2016-05-17 21:29:44,336 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520584090_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:44,336 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:44,336 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E]
2016-05-17 21:29:44,337 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A39DACA3B8C2403EAE500A93F3ECE853/CEFB43705F0A4AC98B68A3373E935621] every/(1/1) ...lyplatform/0_ARGS_RESULTS, VertexName: D1EBB0E187294BC29207099B15BA961E, VertexParallelism: 1, TaskAttemptID:attempt_1463520584090_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A5141C8B035547F2BE4D843B2C7D550C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=863B14D615D84A389641AF4A49132894, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:44,337 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520584090_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:44,337 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520584090_0001_00_000001, stoppedTime=1463520584337, exitStatus=0
2016-05-17 21:29:44,337 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584090_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:44,337 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:44,337 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:44,337 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520584090_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:44,338 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520584090_0001_00_000002
2016-05-17 21:29:44,338 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584090_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:44,338 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584090_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:44,339 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520584090_0001_00_000002
2016-05-17 21:29:44,339 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:44,340 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584090_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:44,340 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:44,341 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520584090_0001_00_000002, launchTime=1463520584340
2016-05-17 21:29:44,341 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520584090_0001_00_000002
2016-05-17 21:29:44,341 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520584090_0001_1_01_000000_0] to container: [container_1463520584090_0001_00_000002]
2016-05-17 21:29:44,342 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584090_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:44,342 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520584090_0001_00_000002 given task: attempt_1463520584090_0001_1_01_000000_0
2016-05-17 21:29:44,342 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520584090_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520584090_0001_1_01_000000_0
2016-05-17 21:29:44,342 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520584090_0001_1_01_000000_0] started. Is using containerId: [container_1463520584090_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:44 Starting to run new task attempt: attempt_1463520584090_0001_1_01_000000_0
2016-05-17 21:29:44,342 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:44,342 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520584090_0001_1_01_000000_0
2016-05-17 21:29:44,342 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:44,342 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D1EBB0E187294BC29207099B15BA961E, taskAttemptId=attempt_1463520584090_0001_1_01_000000_0, startTime=1463520584342, containerId=container_1463520584090_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520584090_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:44,342 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:44,343 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:44,343 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520584090_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:44,343 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:44,343 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:44,343 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520584090_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:44,343 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:44,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A39DACA3B8C2403EAE500A93F3ECE853/CEFB43705F0A4AC98B68A3373E935621] every/(1/1) ...lyplatform/0_ARGS_RESULTS, VertexName: D1EBB0E187294BC29207099B15BA961E, VertexParallelism: 1, TaskAttemptID:attempt_1463520584090_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A5141C8B035547F2BE4D843B2C7D550C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=863B14D615D84A389641AF4A49132894, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:44,343 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:44,343 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520584090_0001_1_01_000000_0
2016-05-17 21:29:44,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:44,343 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:44,343 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520584090_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:44,344 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:44,345 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520584090_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:44,345 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520584090_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:44,345 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520584090_0001_00_000001, Exiting
2016-05-17 21:29:44,345 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520584090_0001_00_000001
2016-05-17 21:29:44,345 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520584090_0001_00_000001
2016-05-17 21:29:44,346 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:44,347 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=863B14D615D84A389641AF4A49132894, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:44,347 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:44,347 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 863B14D615D84A389641AF4A49132894
2016-05-17 21:29:44,347 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:44,347 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 863B14D615D84A389641AF4A49132894
2016-05-17 21:29:44,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A5141C8B035547F2BE4D843B2C7D550C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:44,348 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:44,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A5141C8B035547F2BE4D843B2C7D550C
2016-05-17 21:29:44,350 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:44,350 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:44,351 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D1EBB0E187294BC29207099B15BA961E, ordinal: 1
2016-05-17 21:29:44,351 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D1EBB0E187294BC29207099B15BA961E, mem on start (mb), free: 199, total: 362, max: 672
2016-05-17 21:29:44,351 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:44,351 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:44,351 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520584090_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:44,351 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A5141C8B035547F2BE4D843B2C7D550C
2016-05-17 21:29:44,352 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A5141C8B035547F2BE4D843B2C7D550C
2016-05-17 21:29:44,353 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:44,352 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520584090_0001_00_000001 completed successfully
2016-05-17 21:29:44,353 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584090_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:44,356 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:44,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 863B14D615D84A389641AF4A49132894
2016-05-17 21:29:44,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:44,356 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:44,356 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:44,357 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:44,357 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:44,357 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:44,357 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:44,357 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 863B14D615D84A389641AF4A49132894, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:44,357 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A5141C8B035547F2BE4D843B2C7D550C, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:44,357 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:44,357 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A5141C8B035547F2BE4D843B2C7D550C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:44,357 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:44,357 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A5141C8B035547F2BE4D843B2C7D550C
2016-05-17 21:29:44,357 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:44,359 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:44,359 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:44,359 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A5141C8B035547F2BE4D843B2C7D550C with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:44,359 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:44,360 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:44,360 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:44,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A5141C8B035547F2BE4D843B2C7D550C
2016-05-17 21:29:44,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:44,361 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520584090_0001_1_01_000000_0
2016-05-17 21:29:44,362 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520584090_0001_1_00_000000_0_10213, runDuration: 57173, ]
2016-05-17 21:29:44,363 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520584090_0001_1_00_000000_0_10213] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520584363, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:44,363 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A5141C8B035547F2BE4D843B2C7D550C
2016-05-17 21:29:44,363 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:44,363 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [A5141C8B035547F2BE4D843B2C7D550C] #36 in 1ms
2016-05-17 21:29:44,363 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:44,363 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A5141C8B035547F2BE4D843B2C7D550C] #36, status:false, isInterrupted:false
2016-05-17 21:29:44,364 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:44,364 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 2FF692FC294F4FED83F74E49795349A2
2016-05-17 21:29:44,364 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_RESULTS"], id: 863B14D615D84A389641AF4A49132894
2016-05-17 21:29:44,364 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_RESULTS"]
2016-05-17 21:29:44,364 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 2FF692FC294F4FED83F74E49795349A2, for 1 inputs
2016-05-17 21:29:44,365 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:44,366 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:44,366 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:44,366 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:44,366 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:44,366 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A5141C8B035547F2BE4D843B2C7D550C
2016-05-17 21:29:44,366 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:44,366 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D1EBB0E187294BC29207099B15BA961E, all 1 inputs ready in: 00:00:00.002
2016-05-17 21:29:44,367 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:44,369 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@6d2ce054; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@1d3be2cb
2016-05-17 21:29:44,372 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:44,372 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:44,372 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520584090_0001_1_01_000000_0
2016-05-17 21:29:44,372 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520584090_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:44,373 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520584090_0001_1_01_000000_0
2016-05-17 21:29:44,373 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D1EBB0E187294BC29207099B15BA961E, mem on close (mb), free: 196, total: 362, max: 672
2016-05-17 21:29:44,373 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A5141C8B035547F2BE4D843B2C7D550C
2016-05-17 21:29:44,373 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:44,374 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:44,374 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:44,374 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:44,375 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:44,375 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:44 Completed running task attempt: attempt_1463520584090_0001_1_01_000000_0
2016-05-17 21:29:44,376 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520584090_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:44,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520584366, Process_Duration=7, Process_End_Time=1463520584373, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:44,376 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:44,376 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D1EBB0E187294BC29207099B15BA961E, taskAttemptId=attempt_1463520584090_0001_1_01_000000_0, startTime=1463520584342, finishTime=1463520584376, timeTaken=34, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520584366, Process_Duration=7, Process_End_Time=1463520584373, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:44,376 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:44,376 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:44,377 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520584090_0001_00_000002
2016-05-17 21:29:44,376 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520584090_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:44,377 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:44,377 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520584090_0001_1_01_000000_0
2016-05-17 21:29:44,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:44,378 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:TASK_FINISHED]: vertexName=D1EBB0E187294BC29207099B15BA961E, taskId=task_1463520584090_0001_1_01_000000, startTime=1463520584342, finishTime=1463520584377, timeTaken=35, status=SUCCEEDED, successfulAttemptID=attempt_1463520584090_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520584366, Process_Duration=7, Process_End_Time=1463520584373, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:44,378 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520584090_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:44,379 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E] : 1
2016-05-17 21:29:44,379 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E]
2016-05-17 21:29:44,379 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:VERTEX_FINISHED]: vertexName=D1EBB0E187294BC29207099B15BA961E, vertexId=vertex_1463520584090_0001_1_01, initRequestedTime=1463520584249, initedTime=1463520584255, startRequestedTime=1463520584260, startedTime=1463520584260, finishTime=1463520584379, timeTaken=119, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380108800, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520584366, Process_Duration=7, Process_End_Time=1463520584373, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520584342, firstTasksToStart=[ task_1463520584090_0001_1_01_000000 ], lastTaskFinishTime=1463520584376, lastTasksToFinish=[ task_1463520584090_0001_1_01_000000 ], minTaskDuration=34, maxTaskDuration=34, avgTaskDuration=34.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520584090_0001_1_01_000000 ], longestDurationTasks=[ task_1463520584090_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:44,379 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:44,380 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:44,380 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584090_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:44,380 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520584090_0001_00_000002, stoppedTime=1463520584380, exitStatus=0
2016-05-17 21:29:44,380 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584090_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:44,380 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:44,380 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:44,380 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520584090_0001_1
2016-05-17 21:29:44,381 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520584090_0001_1
2016-05-17 21:29:44,381 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 863B14D615D84A389641AF4A49132894 for vertex: vertex_1463520584090_0001_1_01 [D1EBB0E187294BC29207099B15BA961E]
2016-05-17 21:29:44,387 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520584090_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:44,387 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520584090_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:44,387 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520584090_0001_00_000002, Exiting
2016-05-17 21:29:44,387 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520584090_0001_00_000002
2016-05-17 21:29:44,387 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520584090_0001_00_000002
2016-05-17 21:29:44,388 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520584090_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:44,388 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520584090_0001_00_000002 completed successfully
2016-05-17 21:29:44,388 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520584090_0001_1_00 [A5141C8B035547F2BE4D843B2C7D550C]
2016-05-17 21:29:44,390 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520584090_0001_1, queueSize=0
2016-05-17 21:29:44,390 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:44,390 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584090_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520584090_0001_1, startTime=1463520584248, finishTime=1463520584388, timeTaken=140, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=305, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=17, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=760217600, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041168688, Process_Duration=9, Process_End_Time=2927041168697, Read_Duration=0, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:44,390 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520584090_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:44,391 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520584090_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:44,391 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584090_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:44 Completed Dag: dag_1463520584090_0001_1
2016-05-17 21:29:44,391 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520584090_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:44,391 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520584090_0001_00_000002
2016-05-17 21:29:44,392 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520584090_0001_1_post
2016-05-17 21:29:44,392 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:44,392 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:44,399 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_ARGS_RESULTS, applicationId=application_1463520584090_0001
2016-05-17 21:29:44,400 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:44,400 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:44,400 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:44,400 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:44,400 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:44,401 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:44,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:44,401 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:44,401 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:44,401 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:44,401 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:44,401 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:44,402 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:44,405 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A39DACA3B8C2403EAE500A93F3ECE853/CEFB43705F0A4AC98B68A3373E935621/.tez/application_1463520584090_0001
2016-05-17 21:29:44,406 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_RESULTS/_temporary
2016-05-17 21:29:44,406 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.335
2016-05-17 21:29:44,440 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:44,440 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:44,470 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_RESULTS
2016-05-17 21:29:44,489 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:44,489 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:44,513 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-05-17 21:29:44,516 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-17 21:29:44,516 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:44,516 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:44,516 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_ALL
2016-05-17 21:29:44,536 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:44,537 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:44,565 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 21:29:44,573 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 21:29:44,573 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:44,574 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:44,574 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:44,586 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:44,590 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_REPLACE
2016-05-17 21:29:44,629 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:44,629 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:44,630 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_REPLACE"]
2016-05-17 21:29:44,630 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:44,630 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:44,631 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:44,632 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...lyplatform/0_ARGS_REPLACE
2016-05-17 21:29:44,632 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:44,642 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:44,646 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:44,646 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:44,647 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:44,655 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C9FD7C864F134979BC1B656272FCE033/D2BE61ED2E7C406F86A4D59C43007B8B/.tez/application_1463520584646_0001 doesn't exist and is created
2016-05-17 21:29:44,670 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:44,670 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:44,671 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C9FD7C864F134979BC1B656272FCE033/D2BE61ED2E7C406F86A4D59C43007B8B/.tez/application_1463520584646_0001
2016-05-17 21:29:44,673 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520584646_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:44,682 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:44,682 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:44,683 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:44,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:44,684 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C9FD7C864F134979BC1B656272FCE033/D2BE61ED2E7C406F86A4D59C43007B8B/.tez/application_1463520584646_0001/localmode-log-dir
2016-05-17 21:29:44,684 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C9FD7C864F134979BC1B656272FCE033/D2BE61ED2E7C406F86A4D59C43007B8B/.tez/application_1463520584646_0001/localmode-log-dir/history.txt.appattempt_1463520584646_0001_000000, maxErrors=10
2016-05-17 21:29:44,684 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:44,684 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520584646_0001_000000, appSubmitTime=1463520584672, launchTime=1463520584672
2016-05-17 21:29:44,688 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:44,691 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:44,692 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:34136
2016-05-17 21:29:44,693 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:44,693 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:44,694 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:44,695 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520584646_0001_000000, startTime=1463520584695
2016-05-17 21:29:44,695 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:44,695 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:44,770 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:44,770 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:44,770 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_REPLACE, applicationId=application_1463520584646_0001, dagName=[C9FD7C864F134979BC1B656272FCE033/D2BE61ED2E7C406F86A4D59C43007B8B] every/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-05-17 21:29:44,784 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C9FD7C864F134979BC1B656272FCE033/D2BE61ED2E7C406F86A4D59C43007B8B] every/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-05-17 21:29:44,785 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520584646_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C9FD7C864F134979BC1B656272FCE033/D2BE61ED2E7C406F86A4D59C43007B8B/.tez/application_1463520584646_0001/localmode-log-dir/dag_1463520584646_0001_1.dot
2016-05-17 21:29:44,786 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C9FD7C864F134979BC1B656272FCE033/D2BE61ED2E7C406F86A4D59C43007B8B/.tez/application_1463520584646_0001/localmode-log-dir/dag_1463520584646_0001_1-tez-dag.pb.txt
2016-05-17 21:29:44,792 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520584646_0001_1
2016-05-17 21:29:44,792 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:44,792 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C9FD7C864F134979BC1B656272FCE033/D2BE61ED2E7C406F86A4D59C43007B8B] every/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-05-17 21:29:44 Running Dag: dag_1463520584646_0001_1
2016-05-17 21:29:44,803 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:44,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520584646_0001_1, submitTime=1463520584784
2016-05-17 21:29:44,803 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E969C7EB9B184F60B8CBEAB1B5495C88
2016-05-17 21:29:44,804 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:44,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:44,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520584646_0001_1, initTime=1463520584803
2016-05-17 21:29:44,804 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520584646_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:44,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:DAG_STARTED]: dagID=dag_1463520584646_0001_1, startTime=1463520584804
2016-05-17 21:29:44,805 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:44,805 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520584646_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:44,805 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_REPLACE, applicationId=application_1463520584646_0001, dagName=[C9FD7C864F134979BC1B656272FCE033/D2BE61ED2E7C406F86A4D59C43007B8B] every/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-05-17 21:29:44,805 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D647F7E6093E41AEA7BF07C2F6CCF4B9 : {C44AC5C4A1BC41CA91A7D92F771E69F8={InputName=C44AC5C4A1BC41CA91A7D92F771E69F8}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:44,805 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C44AC5C4A1BC41CA91A7D92F771E69F8, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:44,805 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9]
2016-05-17 21:29:44,805 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9]
2016-05-17 21:29:44,805 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9]
2016-05-17 21:29:44,806 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9]. Starting root input initializers: 1
2016-05-17 21:29:44,806 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:44,806 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:44,806 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88]
2016-05-17 21:29:44,808 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:44,808 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88]
2016-05-17 21:29:44,809 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88]
2016-05-17 21:29:44,809 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88]
2016-05-17 21:29:44,809 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D45558F242F44C41A6CC380347DAA288, vertexId=vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:44,809 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D45558F242F44C41A6CC380347DAA288, vertexId=vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88]
2016-05-17 21:29:44,810 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C44AC5C4A1BC41CA91A7D92F771E69F8 on vertex vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9]
2016-05-17 21:29:44,810 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D45558F242F44C41A6CC380347DAA288, vertexName=E969C7EB9B184F60B8CBEAB1B5495C88, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:44,811 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D45558F242F44C41A6CC380347DAA288, vertexId=vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88]
2016-05-17 21:29:44,812 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:44,812 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E969C7EB9B184F60B8CBEAB1B5495C88, vertexId=vertex_1463520584646_0001_1_01, initRequestedTime=1463520584806, initedTime=1463520584812, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:44,812 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:44,812 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:44,813 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C44AC5C4A1BC41CA91A7D92F771E69F8 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:44,813 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:44,813 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:44,814 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:44,815 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C44AC5C4A1BC41CA91A7D92F771E69F8 on vertex vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9]
2016-05-17 21:29:44,815 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:44,815 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C44AC5C4A1BC41CA91A7D92F771E69F8=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:44,815 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9] parallelism set to 1
2016-05-17 21:29:44,815 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:44,816 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520584646_0001_1_00, eventCount=1
2016-05-17 21:29:44,816 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:44,816 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D647F7E6093E41AEA7BF07C2F6CCF4B9, vertexId=vertex_1463520584646_0001_1_00, initRequestedTime=1463520584805, initedTime=1463520584816, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:44,816 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9] with distanceFromRoot: 0
2016-05-17 21:29:44,816 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:44,817 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D647F7E6093E41AEA7BF07C2F6CCF4B9
2016-05-17 21:29:44,817 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9]
2016-05-17 21:29:44,817 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:44,817 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520584646_0001_1_00, startRequestedTime=1463520584817, startedTime=1463520584817
2016-05-17 21:29:44,817 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:44,818 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:44,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:TASK_STARTED]: vertexName=D647F7E6093E41AEA7BF07C2F6CCF4B9, taskId=task_1463520584646_0001_1_00_000000, scheduledTime=1463520584818, launchTime=1463520584818
2016-05-17 21:29:44,818 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520584646_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:44,818 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520584646_0001_1_00 for vertex: vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88] numStartedSources: 1 numSources: 1
2016-05-17 21:29:44,818 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88] with distanceFromRoot: 1
2016-05-17 21:29:44,819 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9]
2016-05-17 21:29:44,819 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D647F7E6093E41AEA7BF07C2F6CCF4B9 in vertex: E969C7EB9B184F60B8CBEAB1B5495C88
2016-05-17 21:29:44,819 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E969C7EB9B184F60B8CBEAB1B5495C88 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:44,819 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:44,820 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520584646_0001_1_01, startRequestedTime=1463520584819, startedTime=1463520584819
2016-05-17 21:29:44,820 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:44,820 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C9FD7C864F134979BC1B656272FCE033/D2BE61ED2E7C406F86A4D59C43007B8B] every/(1/1) ...lyplatform/0_ARGS_REPLACE, VertexName: D647F7E6093E41AEA7BF07C2F6CCF4B9, VertexParallelism: 1, TaskAttemptID:attempt_1463520584646_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C44AC5C4A1BC41CA91A7D92F771E69F8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E969C7EB9B184F60B8CBEAB1B5495C88, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:44,820 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520584646_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:44,820 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:44,821 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:44,821 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:44,821 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:44,822 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520584646_0001_00_000001
2016-05-17 21:29:44,822 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584646_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:44,823 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520584646_0001, with dag id: dag_1463520584646_0001_1
2016-05-17 21:29:44,823 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520584646_0001_00_000001
2016-05-17 21:29:44,823 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:44,824 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584646_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:44,824 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:44,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520584646_0001_00_000001, launchTime=1463520584824
2016-05-17 21:29:44,825 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520584646_0001_00_000001
2016-05-17 21:29:44,826 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520584646_0001_1_00_000000_0] to container: [container_1463520584646_0001_00_000001]
2016-05-17 21:29:44,826 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584646_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:44,826 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520584646_0001_1_00_000000_0] started. Is using containerId: [container_1463520584646_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:44,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D647F7E6093E41AEA7BF07C2F6CCF4B9, taskAttemptId=attempt_1463520584646_0001_1_00_000000_0, startTime=1463520584826, containerId=container_1463520584646_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520584646_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:44,827 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520584646_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:44,827 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520584646_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:44,827 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:44,827 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520584646_0001_00_000001 given task: attempt_1463520584646_0001_1_00_000000_0
2016-05-17 21:29:44,827 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520584646_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520584646_0001_1_00_000000_0
2016-05-17 21:29:44 Starting to run new task attempt: attempt_1463520584646_0001_1_00_000000_0
2016-05-17 21:29:44,828 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520584646_0001_1_00_000000_0
2016-05-17 21:29:44,828 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:44,828 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:44,828 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:44,828 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:44,828 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:44,828 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:44,828 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C9FD7C864F134979BC1B656272FCE033/D2BE61ED2E7C406F86A4D59C43007B8B] every/(1/1) ...lyplatform/0_ARGS_REPLACE, VertexName: D647F7E6093E41AEA7BF07C2F6CCF4B9, VertexParallelism: 1, TaskAttemptID:attempt_1463520584646_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C44AC5C4A1BC41CA91A7D92F771E69F8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E969C7EB9B184F60B8CBEAB1B5495C88, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:44,828 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:44,829 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520584646_0001_1_00_000000_0
2016-05-17 21:29:44,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:44,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:44,829 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520584646_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:44,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:44,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E969C7EB9B184F60B8CBEAB1B5495C88, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:44,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:44,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E969C7EB9B184F60B8CBEAB1B5495C88
2016-05-17 21:29:44,831 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:44,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C44AC5C4A1BC41CA91A7D92F771E69F8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:44,831 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:44,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:44,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C44AC5C4A1BC41CA91A7D92F771E69F8
2016-05-17 21:29:44,832 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C44AC5C4A1BC41CA91A7D92F771E69F8
2016-05-17 21:29:44,833 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:44,833 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C44AC5C4A1BC41CA91A7D92F771E69F8
2016-05-17 21:29:44,833 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C44AC5C4A1BC41CA91A7D92F771E69F8
2016-05-17 21:29:44,831 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E969C7EB9B184F60B8CBEAB1B5495C88
2016-05-17 21:29:44,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E969C7EB9B184F60B8CBEAB1B5495C88
2016-05-17 21:29:44,836 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D647F7E6093E41AEA7BF07C2F6CCF4B9, ordinal: 0
2016-05-17 21:29:44,836 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D647F7E6093E41AEA7BF07C2F6CCF4B9, mem on start (mb), free: 90, total: 362, max: 672
2016-05-17 21:29:44,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:44,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:44,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:44,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:44,836 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:44,836 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:44,837 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:44,837 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:44,837 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:44,837 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:44,837 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C44AC5C4A1BC41CA91A7D92F771E69F8, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:44,837 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E969C7EB9B184F60B8CBEAB1B5495C88, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:44,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:44,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C44AC5C4A1BC41CA91A7D92F771E69F8 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:44,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:44,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:44,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C44AC5C4A1BC41CA91A7D92F771E69F8
2016-05-17 21:29:44,841 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:44,841 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C44AC5C4A1BC41CA91A7D92F771E69F8
2016-05-17 21:29:44,841 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:44,841 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520584646_0001_1_00_000000_0
2016-05-17 21:29:44,841 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:44,842 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@42243824
2016-05-17 21:29:44,855 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@42243824
2016-05-17 21:29:44,855 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:44,855 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:44,857 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:44,857 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C44AC5C4A1BC41CA91A7D92F771E69F8
2016-05-17 21:29:44,857 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 1413BF8E30F84877AF19717887684291
2016-05-17 21:29:44,857 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 1413BF8E30F84877AF19717887684291
2016-05-17 21:29:44,858 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:44,861 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:44,861 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@3bcb5214; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@2179bdb7; comparator=cascading.tuple.hadoop.util.TupleComparator@3f964a8c; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:44,861 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:44,861 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:44,861 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:44,875 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:44,875 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:44,875 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:44,875 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:44,875 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:44,876 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:44,876 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:44,876 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D647F7E6093E41AEA7BF07C2F6CCF4B9, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:44,878 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520584646_0001_1_00_000000_0
2016-05-17 21:29:44,878 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D647F7E6093E41AEA7BF07C2F6CCF4B9, mem on close (mb), free: 204, total: 363, max: 672
2016-05-17 21:29:44,878 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:44,879 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:44,879 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:44,879 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:44,882 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:44 Completed running task attempt: attempt_1463520584646_0001_1_00_000000_0
2016-05-17 21:29:44,884 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520584646_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:44,885 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520584646_0001_1_00, eventCount=1
2016-05-17 21:29:44,885 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:44,885 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D647F7E6093E41AEA7BF07C2F6CCF4B9, taskAttemptId=attempt_1463520584646_0001_1_00_000000_0, startTime=1463520584826, finishTime=1463520584885, timeTaken=59, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380633088, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520584876, Process_Duration=2, Process_End_Time=1463520584878, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:44,885 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520584646_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:44,885 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520584646_0001_1_00_000000_0
2016-05-17 21:29:44,885 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:44,886 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:TASK_FINISHED]: vertexName=D647F7E6093E41AEA7BF07C2F6CCF4B9, taskId=task_1463520584646_0001_1_00_000000, startTime=1463520584826, finishTime=1463520584885, timeTaken=59, status=SUCCEEDED, successfulAttemptID=attempt_1463520584646_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380633088, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520584876, Process_Duration=2, Process_End_Time=1463520584878, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:44,886 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520584646_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:44,886 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9] : 1
2016-05-17 21:29:44,886 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9]
2016-05-17 21:29:44,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:VERTEX_FINISHED]: vertexName=D647F7E6093E41AEA7BF07C2F6CCF4B9, vertexId=vertex_1463520584646_0001_1_00, initRequestedTime=1463520584805, initedTime=1463520584816, startRequestedTime=1463520584817, startedTime=1463520584817, finishTime=1463520584886, timeTaken=69, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380633088, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520584876, Process_Duration=2, Process_End_Time=1463520584878, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520584826, firstTasksToStart=[ task_1463520584646_0001_1_00_000000 ], lastTaskFinishTime=1463520584885, lastTasksToFinish=[ task_1463520584646_0001_1_00_000000 ], minTaskDuration=59, maxTaskDuration=59, avgTaskDuration=59.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520584646_0001_1_00_000000 ], longestDurationTasks=[ task_1463520584646_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:44,887 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:44,888 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584646_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:44,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520584646_0001_00_000001, stoppedTime=1463520584888, exitStatus=0
2016-05-17 21:29:44,888 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584646_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:44,888 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88] attempt: attempt_1463520584646_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:44,888 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E969C7EB9B184F60B8CBEAB1B5495C88
2016-05-17 21:29:44,888 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88]
2016-05-17 21:29:44,888 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:44,888 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520584646_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:44,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:TASK_STARTED]: vertexName=E969C7EB9B184F60B8CBEAB1B5495C88, taskId=task_1463520584646_0001_1_01_000000, scheduledTime=1463520584888, launchTime=1463520584888
2016-05-17 21:29:44,889 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520584646_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:44,889 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88]
2016-05-17 21:29:44,889 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584646_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:44,889 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C9FD7C864F134979BC1B656272FCE033/D2BE61ED2E7C406F86A4D59C43007B8B] every/(1/1) ...lyplatform/0_ARGS_REPLACE, VertexName: E969C7EB9B184F60B8CBEAB1B5495C88, VertexParallelism: 1, TaskAttemptID:attempt_1463520584646_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D647F7E6093E41AEA7BF07C2F6CCF4B9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D45558F242F44C41A6CC380347DAA288, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:44,889 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520584646_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:44,889 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380633088, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520584876, Process_Duration=2, Process_End_Time=1463520584878, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:44,890 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:44,890 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:44,890 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520584646_0001_00_000001
2016-05-17 21:29:44,890 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520584646_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:44,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:44,891 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:44,891 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:44,891 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:44,892 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520584646_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:44,892 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520584646_0001_00_000001, Exiting
2016-05-17 21:29:44,892 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520584646_0001_00_000001
2016-05-17 21:29:44,892 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520584646_0001_00_000001
2016-05-17 21:29:44,893 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520584646_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:44,893 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520584646_0001_00_000001 completed successfully
2016-05-17 21:29:44,893 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584646_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:44,894 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:44,895 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520584646_0001_00_000002
2016-05-17 21:29:44,895 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584646_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:44,896 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520584646_0001_00_000002
2016-05-17 21:29:44,897 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:44,899 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584646_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:44,900 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:44,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520584646_0001_00_000002, launchTime=1463520584900
2016-05-17 21:29:44,901 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520584646_0001_00_000002
2016-05-17 21:29:44,902 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520584646_0001_1_01_000000_0] to container: [container_1463520584646_0001_00_000002]
2016-05-17 21:29:44,902 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584646_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:44,903 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520584646_0001_1_01_000000_0] started. Is using containerId: [container_1463520584646_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:44,903 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E969C7EB9B184F60B8CBEAB1B5495C88, taskAttemptId=attempt_1463520584646_0001_1_01_000000_0, startTime=1463520584902, containerId=container_1463520584646_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520584646_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:44,903 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520584646_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:44,903 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520584646_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:44,903 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:44,904 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520584646_0001_00_000002 given task: attempt_1463520584646_0001_1_01_000000_0
2016-05-17 21:29:44,904 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520584646_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520584646_0001_1_01_000000_0
2016-05-17 21:29:44 Starting to run new task attempt: attempt_1463520584646_0001_1_01_000000_0
2016-05-17 21:29:44,905 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520584646_0001_1_01_000000_0
2016-05-17 21:29:44,905 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:44,905 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:44,905 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:44,905 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:44,905 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:44,906 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:44,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C9FD7C864F134979BC1B656272FCE033/D2BE61ED2E7C406F86A4D59C43007B8B] every/(1/1) ...lyplatform/0_ARGS_REPLACE, VertexName: E969C7EB9B184F60B8CBEAB1B5495C88, VertexParallelism: 1, TaskAttemptID:attempt_1463520584646_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D647F7E6093E41AEA7BF07C2F6CCF4B9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D45558F242F44C41A6CC380347DAA288, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:44,906 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:44,908 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520584646_0001_1_01_000000_0
2016-05-17 21:29:44,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:44,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:44,909 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520584646_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:44,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D647F7E6093E41AEA7BF07C2F6CCF4B9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:44,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:44,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D647F7E6093E41AEA7BF07C2F6CCF4B9
2016-05-17 21:29:44,913 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:44,913 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:44,914 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D647F7E6093E41AEA7BF07C2F6CCF4B9
2016-05-17 21:29:44,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D647F7E6093E41AEA7BF07C2F6CCF4B9
2016-05-17 21:29:44,915 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:44,917 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:44,922 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E969C7EB9B184F60B8CBEAB1B5495C88, ordinal: 1
2016-05-17 21:29:44,923 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E969C7EB9B184F60B8CBEAB1B5495C88, mem on start (mb), free: 199, total: 363, max: 672
2016-05-17 21:29:44,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:44,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:44,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:44,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D45558F242F44C41A6CC380347DAA288, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:44,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:44,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D45558F242F44C41A6CC380347DAA288
2016-05-17 21:29:44,926 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:44,926 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D45558F242F44C41A6CC380347DAA288
2016-05-17 21:29:44,935 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:44,935 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D45558F242F44C41A6CC380347DAA288
2016-05-17 21:29:44,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:44,936 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:44,936 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:44,936 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:44,936 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:44,936 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:44,936 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:44,936 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D647F7E6093E41AEA7BF07C2F6CCF4B9, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:44,936 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D45558F242F44C41A6CC380347DAA288, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:44,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:44,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D647F7E6093E41AEA7BF07C2F6CCF4B9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:44,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D647F7E6093E41AEA7BF07C2F6CCF4B9
2016-05-17 21:29:44,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:44,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:44,937 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:44,938 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:44,939 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D647F7E6093E41AEA7BF07C2F6CCF4B9 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:44,939 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:44,939 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:44,939 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:44,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D647F7E6093E41AEA7BF07C2F6CCF4B9
2016-05-17 21:29:44,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:44,942 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520584646_0001_1_01_000000_0
2016-05-17 21:29:44,944 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:44,944 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520584646_0001_1_00_000000_0_10218, runDuration: 53452, ]
2016-05-17 21:29:44,945 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 1413BF8E30F84877AF19717887684291
2016-05-17 21:29:44,945 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_REPLACE"], id: D45558F242F44C41A6CC380347DAA288
2016-05-17 21:29:44,945 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_REPLACE"]
2016-05-17 21:29:44,945 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 1413BF8E30F84877AF19717887684291, for 1 inputs
2016-05-17 21:29:44,946 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520584646_0001_1_00_000000_0_10218] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520584946, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:44,947 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D647F7E6093E41AEA7BF07C2F6CCF4B9
2016-05-17 21:29:44,947 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:44,948 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D647F7E6093E41AEA7BF07C2F6CCF4B9] #37 in 2ms
2016-05-17 21:29:44,948 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D647F7E6093E41AEA7BF07C2F6CCF4B9] #37, status:false, isInterrupted:false
2016-05-17 21:29:44,949 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:44,949 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:44,950 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:44,950 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:44,950 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:44,950 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:44,950 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D647F7E6093E41AEA7BF07C2F6CCF4B9
2016-05-17 21:29:44,950 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:44,950 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E969C7EB9B184F60B8CBEAB1B5495C88, all 1 inputs ready in: 00:00:00.004
2016-05-17 21:29:44,952 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:44,953 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@4ddf9efb; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@143e98ae
2016-05-17 21:29:44,956 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:44,956 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:44,957 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520584646_0001_1_01_000000_0
2016-05-17 21:29:44,957 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520584646_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:44,957 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520584646_0001_1_01_000000_0
2016-05-17 21:29:44,957 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E969C7EB9B184F60B8CBEAB1B5495C88, mem on close (mb), free: 196, total: 363, max: 672
2016-05-17 21:29:44,958 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D647F7E6093E41AEA7BF07C2F6CCF4B9
2016-05-17 21:29:44,958 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:44,960 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:44,960 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:44,960 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:44,960 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:44,960 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:44 Completed running task attempt: attempt_1463520584646_0001_1_01_000000_0
2016-05-17 21:29:44,961 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520584646_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:44,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E969C7EB9B184F60B8CBEAB1B5495C88, taskAttemptId=attempt_1463520584646_0001_1_01_000000_0, startTime=1463520584902, finishTime=1463520584962, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=143, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380633088, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520584950, Process_Duration=7, Process_End_Time=1463520584957, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:44,962 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520584646_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:44,962 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520584646_0001_1_01_000000_0
2016-05-17 21:29:44,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:TASK_FINISHED]: vertexName=E969C7EB9B184F60B8CBEAB1B5495C88, taskId=task_1463520584646_0001_1_01_000000, startTime=1463520584902, finishTime=1463520584962, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1463520584646_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=143, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380633088, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520584950, Process_Duration=7, Process_End_Time=1463520584957, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:44,963 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520584646_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:44,963 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88] : 1
2016-05-17 21:29:44,963 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88]
2016-05-17 21:29:44,963 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:44,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:VERTEX_FINISHED]: vertexName=E969C7EB9B184F60B8CBEAB1B5495C88, vertexId=vertex_1463520584646_0001_1_01, initRequestedTime=1463520584806, initedTime=1463520584812, startRequestedTime=1463520584819, startedTime=1463520584819, finishTime=1463520584963, timeTaken=144, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=143, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380633088, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520584950, Process_Duration=7, Process_End_Time=1463520584957, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520584902, firstTasksToStart=[ task_1463520584646_0001_1_01_000000 ], lastTaskFinishTime=1463520584962, lastTasksToFinish=[ task_1463520584646_0001_1_01_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520584646_0001_1_01_000000 ], longestDurationTasks=[ task_1463520584646_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:44,964 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:44,964 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:44,964 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:44,964 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520584646_0001_1
2016-05-17 21:29:44,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520584646_0001_1
2016-05-17 21:29:44,964 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:44,964 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520584646_0001_1_00 [D647F7E6093E41AEA7BF07C2F6CCF4B9]
2016-05-17 21:29:44,965 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D45558F242F44C41A6CC380347DAA288 for vertex: vertex_1463520584646_0001_1_01 [E969C7EB9B184F60B8CBEAB1B5495C88]
2016-05-17 21:29:44,966 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:44,973 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:44,973 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:44,975 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520584646_0001_1, queueSize=0
2016-05-17 21:29:44,975 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:44,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520584646_0001_1, startTime=1463520584804, finishTime=1463520584973, timeTaken=169, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=356, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=761266176, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041169826, Process_Duration=9, Process_End_Time=2927041169835, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:44,975 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520584646_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:44,975 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520584646_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:44,976 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584646_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:44,976 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520584646_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520584646_0001_00_000002, stoppedTime=1463520584976, exitStatus=0
2016-05-17 21:29:44,976 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:44,976 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584646_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:44 Completed Dag: dag_1463520584646_0001_1
2016-05-17 21:29:44,976 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520584646_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:44,976 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520584646_0001_1_post
2016-05-17 21:29:44,976 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:44,976 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:44,977 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=143, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380633088, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520584950, Process_Duration=7, Process_End_Time=1463520584957, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:44,977 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520584646_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:44,977 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:44,977 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:44,977 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584646_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:44,977 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520584646_0001_00_000002
2016-05-17 21:29:44,977 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520584646_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:44,977 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520584646_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:44,978 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520584646_0001_00_000002, Exiting
2016-05-17 21:29:44,978 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520584646_0001_00_000002
2016-05-17 21:29:44,978 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520584646_0001_00_000002
2016-05-17 21:29:44,978 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520584646_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:44,978 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520584646_0001_00_000002 completed successfully
2016-05-17 21:29:44,978 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520584646_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:44,982 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_ARGS_REPLACE, applicationId=application_1463520584646_0001
2016-05-17 21:29:44,984 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:44,984 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:44,984 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:44,984 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:44,984 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:44,985 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:44,985 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:44,986 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:44,986 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:44,986 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:44,986 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:44,987 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:44,987 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:44,992 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C9FD7C864F134979BC1B656272FCE033/D2BE61ED2E7C406F86A4D59C43007B8B/.tez/application_1463520584646_0001
2016-05-17 21:29:44,993 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_REPLACE/_temporary
2016-05-17 21:29:44,993 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.363
2016-05-17 21:29:45,060 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_REPLACE
2016-05-17 21:29:45,092 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:45,092 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:45,129 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 21:29:45,138 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-17 21:29:45,138 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:45,139 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:45,139 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:45,158 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:45,164 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_SWAP
2016-05-17 21:29:45,204 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:45,205 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:45,205 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_SWAP"]
2016-05-17 21:29:45,205 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:45,205 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:45,205 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:45,206 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...emblyplatform/0_ARGS_SWAP
2016-05-17 21:29:45,206 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:45,216 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:45,220 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:45,221 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:45,221 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:45,230 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0B2779ADBF1F496585B91CE8131D36F6/CC5243738A704BAAB076A2EDCEFAEDFA/.tez/application_1463520585220_0001 doesn't exist and is created
2016-05-17 21:29:45,243 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:45,244 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:45,244 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0B2779ADBF1F496585B91CE8131D36F6/CC5243738A704BAAB076A2EDCEFAEDFA/.tez/application_1463520585220_0001
2016-05-17 21:29:45,246 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520585220_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:45,256 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:45,256 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:45,257 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:45,257 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:45,258 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0B2779ADBF1F496585B91CE8131D36F6/CC5243738A704BAAB076A2EDCEFAEDFA/.tez/application_1463520585220_0001/localmode-log-dir
2016-05-17 21:29:45,258 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0B2779ADBF1F496585B91CE8131D36F6/CC5243738A704BAAB076A2EDCEFAEDFA/.tez/application_1463520585220_0001/localmode-log-dir/history.txt.appattempt_1463520585220_0001_000000, maxErrors=10
2016-05-17 21:29:45,258 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:45,258 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520585220_0001_000000, appSubmitTime=1463520585246, launchTime=1463520585246
2016-05-17 21:29:45,263 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:45,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:45,266 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:45,266 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:45,267 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:60311
2016-05-17 21:29:45,270 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:45,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520585220_0001_000000, startTime=1463520585271
2016-05-17 21:29:45,271 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:45,271 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:45,344 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:45,344 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:45,344 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...emblyplatform/0_ARGS_SWAP, applicationId=application_1463520585220_0001, dagName=[0B2779ADBF1F496585B91CE8131D36F6/CC5243738A704BAAB076A2EDCEFAEDFA] every/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-05-17 21:29:45,358 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [0B2779ADBF1F496585B91CE8131D36F6/CC5243738A704BAAB076A2EDCEFAEDFA] every/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-05-17 21:29:45,359 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520585220_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0B2779ADBF1F496585B91CE8131D36F6/CC5243738A704BAAB076A2EDCEFAEDFA/.tez/application_1463520585220_0001/localmode-log-dir/dag_1463520585220_0001_1.dot
2016-05-17 21:29:45,360 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0B2779ADBF1F496585B91CE8131D36F6/CC5243738A704BAAB076A2EDCEFAEDFA/.tez/application_1463520585220_0001/localmode-log-dir/dag_1463520585220_0001_1-tez-dag.pb.txt
2016-05-17 21:29:45,365 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520585220_0001_1
2016-05-17 21:29:45,365 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:45,365 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [0B2779ADBF1F496585B91CE8131D36F6/CC5243738A704BAAB076A2EDCEFAEDFA] every/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-05-17 21:29:45 Running Dag: dag_1463520585220_0001_1
2016-05-17 21:29:45,375 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520585220_0001_1, submitTime=1463520585358
2016-05-17 21:29:45,375 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:45,375 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FCFD7BBC133548FBADD722570DE14202
2016-05-17 21:29:45,375 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:45,376 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520585220_0001_1, initTime=1463520585375
2016-05-17 21:29:45,376 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:45,376 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520585220_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:45,376 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:DAG_STARTED]: dagID=dag_1463520585220_0001_1, startTime=1463520585376
2016-05-17 21:29:45,376 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:45,376 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520585220_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:45,377 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: ADA5E9343780468A8B98F9C6219D5D24 : {22862AE19DF34E90A41A29126C38D7A5={InputName=22862AE19DF34E90A41A29126C38D7A5}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:45,377 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 22862AE19DF34E90A41A29126C38D7A5, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:45,377 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24]
2016-05-17 21:29:45,377 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...emblyplatform/0_ARGS_SWAP, applicationId=application_1463520585220_0001, dagName=[0B2779ADBF1F496585B91CE8131D36F6/CC5243738A704BAAB076A2EDCEFAEDFA] every/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-05-17 21:29:45,377 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:45,377 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24]
2016-05-17 21:29:45,377 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24]
2016-05-17 21:29:45,377 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24]. Starting root input initializers: 1
2016-05-17 21:29:45,378 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:45,378 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202]
2016-05-17 21:29:45,380 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:45,380 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202]
2016-05-17 21:29:45,380 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202]
2016-05-17 21:29:45,380 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202]
2016-05-17 21:29:45,380 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=C54E2711C04648D1B1301EF1274F89F6, vertexId=vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:45,380 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=C54E2711C04648D1B1301EF1274F89F6, vertexId=vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202]
2016-05-17 21:29:45,380 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 22862AE19DF34E90A41A29126C38D7A5 on vertex vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24]
2016-05-17 21:29:45,382 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=C54E2711C04648D1B1301EF1274F89F6, vertexName=FCFD7BBC133548FBADD722570DE14202, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:45,382 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=C54E2711C04648D1B1301EF1274F89F6, vertexId=vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202]
2016-05-17 21:29:45,383 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:45,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FCFD7BBC133548FBADD722570DE14202, vertexId=vertex_1463520585220_0001_1_01, initRequestedTime=1463520585378, initedTime=1463520585383, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:45,383 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:45,384 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:45,384 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 22862AE19DF34E90A41A29126C38D7A5 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:45,384 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:45,384 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:45,385 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:45,386 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 22862AE19DF34E90A41A29126C38D7A5 on vertex vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24]
2016-05-17 21:29:45,386 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:45,386 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {22862AE19DF34E90A41A29126C38D7A5=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:45,386 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24] parallelism set to 1
2016-05-17 21:29:45,386 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:45,387 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520585220_0001_1_00, eventCount=1
2016-05-17 21:29:45,387 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:45,387 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ADA5E9343780468A8B98F9C6219D5D24, vertexId=vertex_1463520585220_0001_1_00, initRequestedTime=1463520585377, initedTime=1463520585387, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:45,387 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24] with distanceFromRoot: 0
2016-05-17 21:29:45,388 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:45,388 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in ADA5E9343780468A8B98F9C6219D5D24
2016-05-17 21:29:45,388 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24]
2016-05-17 21:29:45,388 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:45,388 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520585220_0001_1_00, startRequestedTime=1463520585388, startedTime=1463520585388
2016-05-17 21:29:45,389 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:45,389 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:45,389 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:TASK_STARTED]: vertexName=ADA5E9343780468A8B98F9C6219D5D24, taskId=task_1463520585220_0001_1_00_000000, scheduledTime=1463520585389, launchTime=1463520585389
2016-05-17 21:29:45,389 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520585220_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:45,389 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520585220_0001_1_00 for vertex: vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202] numStartedSources: 1 numSources: 1
2016-05-17 21:29:45,390 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202] with distanceFromRoot: 1
2016-05-17 21:29:45,390 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24]
2016-05-17 21:29:45,390 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: ADA5E9343780468A8B98F9C6219D5D24 in vertex: FCFD7BBC133548FBADD722570DE14202
2016-05-17 21:29:45,390 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FCFD7BBC133548FBADD722570DE14202 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:45,391 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:45,391 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520585220_0001_1_01, startRequestedTime=1463520585390, startedTime=1463520585390
2016-05-17 21:29:45,391 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:45,391 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0B2779ADBF1F496585B91CE8131D36F6/CC5243738A704BAAB076A2EDCEFAEDFA] every/(1/1) ...emblyplatform/0_ARGS_SWAP, VertexName: ADA5E9343780468A8B98F9C6219D5D24, VertexParallelism: 1, TaskAttemptID:attempt_1463520585220_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=22862AE19DF34E90A41A29126C38D7A5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FCFD7BBC133548FBADD722570DE14202, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:45,391 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520585220_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:45,391 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:45,392 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:45,392 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:45,392 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:45,393 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520585220_0001_00_000001
2016-05-17 21:29:45,393 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585220_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:45,394 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520585220_0001_00_000001
2016-05-17 21:29:45,394 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:45,394 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520585220_0001, with dag id: dag_1463520585220_0001_1
2016-05-17 21:29:45,394 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585220_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:45,395 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:45,395 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520585220_0001_00_000001, launchTime=1463520585394
2016-05-17 21:29:45,395 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520585220_0001_00_000001
2016-05-17 21:29:45,396 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520585220_0001_1_00_000000_0] to container: [container_1463520585220_0001_00_000001]
2016-05-17 21:29:45,396 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585220_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:45,396 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520585220_0001_00_000001 given task: attempt_1463520585220_0001_1_00_000000_0
2016-05-17 21:29:45,396 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520585220_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520585220_0001_1_00_000000_0
2016-05-17 21:29:45,396 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520585220_0001_1_00_000000_0] started. Is using containerId: [container_1463520585220_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:45,396 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ADA5E9343780468A8B98F9C6219D5D24, taskAttemptId=attempt_1463520585220_0001_1_00_000000_0, startTime=1463520585396, containerId=container_1463520585220_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520585220_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:45 Starting to run new task attempt: attempt_1463520585220_0001_1_00_000000_0
2016-05-17 21:29:45,397 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520585220_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:45,397 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520585220_0001_1_00_000000_0
2016-05-17 21:29:45,397 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:45,397 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:45,397 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:45,397 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520585220_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:45,397 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:45,397 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:45,397 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:45,397 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:45,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0B2779ADBF1F496585B91CE8131D36F6/CC5243738A704BAAB076A2EDCEFAEDFA] every/(1/1) ...emblyplatform/0_ARGS_SWAP, VertexName: ADA5E9343780468A8B98F9C6219D5D24, VertexParallelism: 1, TaskAttemptID:attempt_1463520585220_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=22862AE19DF34E90A41A29126C38D7A5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FCFD7BBC133548FBADD722570DE14202, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:45,398 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:45,399 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520585220_0001_1_00_000000_0
2016-05-17 21:29:45,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:45,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:45,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=22862AE19DF34E90A41A29126C38D7A5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:45,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:45,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 22862AE19DF34E90A41A29126C38D7A5
2016-05-17 21:29:45,401 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 22862AE19DF34E90A41A29126C38D7A5
2016-05-17 21:29:45,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:45,401 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520585220_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:45,402 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:45,402 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 22862AE19DF34E90A41A29126C38D7A5
2016-05-17 21:29:45,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 22862AE19DF34E90A41A29126C38D7A5
2016-05-17 21:29:45,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FCFD7BBC133548FBADD722570DE14202, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:45,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:45,404 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:45,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FCFD7BBC133548FBADD722570DE14202
2016-05-17 21:29:45,405 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:45,405 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FCFD7BBC133548FBADD722570DE14202
2016-05-17 21:29:45,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FCFD7BBC133548FBADD722570DE14202
2016-05-17 21:29:45,407 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ADA5E9343780468A8B98F9C6219D5D24, ordinal: 0
2016-05-17 21:29:45,407 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADA5E9343780468A8B98F9C6219D5D24, mem on start (mb), free: 104, total: 363, max: 672
2016-05-17 21:29:45,408 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:45,408 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:45,408 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:45,408 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:45,408 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:45,408 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:45,408 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:45,408 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:45,408 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:45,408 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:45,409 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 22862AE19DF34E90A41A29126C38D7A5, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:45,409 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FCFD7BBC133548FBADD722570DE14202, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:45,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:45,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 22862AE19DF34E90A41A29126C38D7A5 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:45,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 22862AE19DF34E90A41A29126C38D7A5
2016-05-17 21:29:45,409 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:45,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 22862AE19DF34E90A41A29126C38D7A5
2016-05-17 21:29:45,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:45,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:45,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:45,410 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520585220_0001_1_00_000000_0
2016-05-17 21:29:45,410 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:45,411 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@78c66247
2016-05-17 21:29:45,412 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@78c66247
2016-05-17 21:29:45,412 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:45,412 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:45,413 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:45,413 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 22862AE19DF34E90A41A29126C38D7A5
2016-05-17 21:29:45,413 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 8D6E2A6C62A34C058F2F7EAB50306506
2016-05-17 21:29:45,414 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 8D6E2A6C62A34C058F2F7EAB50306506
2016-05-17 21:29:45,415 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:45,416 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:45,417 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@7d8dd7da; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@6b56699b; comparator=cascading.tuple.hadoop.util.TupleComparator@762f762a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:45,417 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:45,417 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:45,417 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:45,441 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:45,441 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:45,441 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:45,441 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:45,441 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:45,442 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:45,442 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:45,444 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ADA5E9343780468A8B98F9C6219D5D24, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:45,446 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520585220_0001_1_00_000000_0
2016-05-17 21:29:45,446 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADA5E9343780468A8B98F9C6219D5D24, mem on close (mb), free: 206, total: 364, max: 672
2016-05-17 21:29:45,446 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:45,447 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:45,447 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:45,447 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:45,450 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:45 Completed running task attempt: attempt_1463520585220_0001_1_00_000000_0
2016-05-17 21:29:45,452 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520585220_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:45,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520585220_0001_1_00, eventCount=1
2016-05-17 21:29:45,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:45,454 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ADA5E9343780468A8B98F9C6219D5D24, taskAttemptId=attempt_1463520585220_0001_1_00_000000_0, startTime=1463520585396, finishTime=1463520585453, timeTaken=57, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520585444, Process_Duration=2, Process_End_Time=1463520585446, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:45,454 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520585220_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:45,454 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520585220_0001_1_00_000000_0
2016-05-17 21:29:45,454 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:45,454 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:45,454 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:TASK_FINISHED]: vertexName=ADA5E9343780468A8B98F9C6219D5D24, taskId=task_1463520585220_0001_1_00_000000, startTime=1463520585396, finishTime=1463520585454, timeTaken=58, status=SUCCEEDED, successfulAttemptID=attempt_1463520585220_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520585444, Process_Duration=2, Process_End_Time=1463520585446, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:45,454 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520585220_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:45,455 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24] : 1
2016-05-17 21:29:45,455 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24]
2016-05-17 21:29:45,456 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:VERTEX_FINISHED]: vertexName=ADA5E9343780468A8B98F9C6219D5D24, vertexId=vertex_1463520585220_0001_1_00, initRequestedTime=1463520585377, initedTime=1463520585387, startRequestedTime=1463520585388, startedTime=1463520585388, finishTime=1463520585455, timeTaken=67, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520585444, Process_Duration=2, Process_End_Time=1463520585446, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520585396, firstTasksToStart=[ task_1463520585220_0001_1_00_000000 ], lastTaskFinishTime=1463520585453, lastTasksToFinish=[ task_1463520585220_0001_1_00_000000 ], minTaskDuration=57, maxTaskDuration=57, avgTaskDuration=57.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520585220_0001_1_00_000000 ], longestDurationTasks=[ task_1463520585220_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:45,456 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:45,456 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:45,457 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585220_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:45,457 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520585220_0001_00_000001, stoppedTime=1463520585457, exitStatus=0
2016-05-17 21:29:45,457 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585220_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:45,457 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202] attempt: attempt_1463520585220_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:45,457 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FCFD7BBC133548FBADD722570DE14202
2016-05-17 21:29:45,457 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202]
2016-05-17 21:29:45,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520585444, Process_Duration=2, Process_End_Time=1463520585446, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:45,457 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:45,458 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:45,458 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:45,458 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:TASK_STARTED]: vertexName=FCFD7BBC133548FBADD722570DE14202, taskId=task_1463520585220_0001_1_01_000000, scheduledTime=1463520585458, launchTime=1463520585458
2016-05-17 21:29:45,458 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520585220_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:45,458 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:45,458 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520585220_0001_00_000001
2016-05-17 21:29:45,458 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:45,458 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520585220_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:45,458 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520585220_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:45,458 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520585220_0001_00_000001, Exiting
2016-05-17 21:29:45,459 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520585220_0001_00_000001
2016-05-17 21:29:45,459 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520585220_0001_00_000001
2016-05-17 21:29:45,458 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520585220_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:45,459 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202]
2016-05-17 21:29:45,459 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585220_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:45,459 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0B2779ADBF1F496585B91CE8131D36F6/CC5243738A704BAAB076A2EDCEFAEDFA] every/(1/1) ...emblyplatform/0_ARGS_SWAP, VertexName: FCFD7BBC133548FBADD722570DE14202, VertexParallelism: 1, TaskAttemptID:attempt_1463520585220_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ADA5E9343780468A8B98F9C6219D5D24, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C54E2711C04648D1B1301EF1274F89F6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:45,459 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520585220_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:45,460 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:45,460 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520585220_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:45,460 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520585220_0001_00_000001 completed successfully
2016-05-17 21:29:45,460 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585220_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:45,461 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520585220_0001_00_000002
2016-05-17 21:29:45,461 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585220_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:45,462 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520585220_0001_00_000002
2016-05-17 21:29:45,463 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:45,465 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585220_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:45,465 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:45,466 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520585220_0001_00_000002, launchTime=1463520585465
2016-05-17 21:29:45,466 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520585220_0001_00_000002
2016-05-17 21:29:45,467 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520585220_0001_1_01_000000_0] to container: [container_1463520585220_0001_00_000002]
2016-05-17 21:29:45,467 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585220_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:45,467 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520585220_0001_1_01_000000_0] started. Is using containerId: [container_1463520585220_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:45,467 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FCFD7BBC133548FBADD722570DE14202, taskAttemptId=attempt_1463520585220_0001_1_01_000000_0, startTime=1463520585467, containerId=container_1463520585220_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520585220_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:45,467 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:45,467 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520585220_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:45,467 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520585220_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:45,469 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520585220_0001_00_000002 given task: attempt_1463520585220_0001_1_01_000000_0
2016-05-17 21:29:45,469 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520585220_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520585220_0001_1_01_000000_0
2016-05-17 21:29:45 Starting to run new task attempt: attempt_1463520585220_0001_1_01_000000_0
2016-05-17 21:29:45,470 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520585220_0001_1_01_000000_0
2016-05-17 21:29:45,470 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:45,470 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:45,470 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:45,470 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:45,470 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:45,471 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:45,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0B2779ADBF1F496585B91CE8131D36F6/CC5243738A704BAAB076A2EDCEFAEDFA] every/(1/1) ...emblyplatform/0_ARGS_SWAP, VertexName: FCFD7BBC133548FBADD722570DE14202, VertexParallelism: 1, TaskAttemptID:attempt_1463520585220_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ADA5E9343780468A8B98F9C6219D5D24, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C54E2711C04648D1B1301EF1274F89F6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:45,471 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:45,471 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520585220_0001_1_01_000000_0
2016-05-17 21:29:45,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:45,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:45,472 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520585220_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:45,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:45,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C54E2711C04648D1B1301EF1274F89F6, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:45,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:45,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C54E2711C04648D1B1301EF1274F89F6
2016-05-17 21:29:45,473 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:45,474 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: C54E2711C04648D1B1301EF1274F89F6
2016-05-17 21:29:45,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=ADA5E9343780468A8B98F9C6219D5D24, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:45,475 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:45,475 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:45,475 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: ADA5E9343780468A8B98F9C6219D5D24
2016-05-17 21:29:45,476 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:45,476 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:45,477 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: ADA5E9343780468A8B98F9C6219D5D24
2016-05-17 21:29:45,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: ADA5E9343780468A8B98F9C6219D5D24
2016-05-17 21:29:45,479 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FCFD7BBC133548FBADD722570DE14202, ordinal: 1
2016-05-17 21:29:45,480 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FCFD7BBC133548FBADD722570DE14202, mem on start (mb), free: 200, total: 364, max: 672
2016-05-17 21:29:45,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:45,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:45,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:45,483 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:45,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C54E2711C04648D1B1301EF1274F89F6
2016-05-17 21:29:45,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:45,483 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:45,483 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:45,484 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:45,484 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:45,484 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:45,484 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:45,484 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C54E2711C04648D1B1301EF1274F89F6, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:45,484 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, ADA5E9343780468A8B98F9C6219D5D24, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:45,484 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:45,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: ADA5E9343780468A8B98F9C6219D5D24 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:45,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: ADA5E9343780468A8B98F9C6219D5D24
2016-05-17 21:29:45,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:45,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:45,485 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:45,486 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:45,486 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: ADA5E9343780468A8B98F9C6219D5D24 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:45,486 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:45,486 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:45,486 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:45,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: ADA5E9343780468A8B98F9C6219D5D24
2016-05-17 21:29:45,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:45,488 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520585220_0001_1_01_000000_0
2016-05-17 21:29:45,489 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520585220_0001_1_00_000000_0_10225, runDuration: 47549, ]
2016-05-17 21:29:45,490 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520585220_0001_1_00_000000_0_10225] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520585490, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:45,490 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : ADA5E9343780468A8B98F9C6219D5D24
2016-05-17 21:29:45,490 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:45,491 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 8D6E2A6C62A34C058F2F7EAB50306506
2016-05-17 21:29:45,491 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:45,491 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_SWAP"], id: C54E2711C04648D1B1301EF1274F89F6
2016-05-17 21:29:45,491 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_SWAP"]
2016-05-17 21:29:45,491 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [ADA5E9343780468A8B98F9C6219D5D24] #38 in 2ms
2016-05-17 21:29:45,491 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 8D6E2A6C62A34C058F2F7EAB50306506, for 1 inputs
2016-05-17 21:29:45,492 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [ADA5E9343780468A8B98F9C6219D5D24] #38, status:false, isInterrupted:false
2016-05-17 21:29:45,492 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:45,493 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:45,493 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:45,493 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:45,493 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:45,494 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:45,494 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : ADA5E9343780468A8B98F9C6219D5D24
2016-05-17 21:29:45,494 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:45,494 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FCFD7BBC133548FBADD722570DE14202, all 1 inputs ready in: 00:00:00.002
2016-05-17 21:29:45,495 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:45,497 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@7d453291; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@49fb0dd7
2016-05-17 21:29:45,501 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:45,501 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:45,501 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:45,501 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:45,501 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520585220_0001_1_01_000000_0
2016-05-17 21:29:45,502 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520585220_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:45,502 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520585220_0001_1_01_000000_0
2016-05-17 21:29:45,502 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FCFD7BBC133548FBADD722570DE14202, mem on close (mb), free: 197, total: 364, max: 672
2016-05-17 21:29:45,502 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: ADA5E9343780468A8B98F9C6219D5D24
2016-05-17 21:29:45,502 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:45,503 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:45,503 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:45,503 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:45,503 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:45,503 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:45 Completed running task attempt: attempt_1463520585220_0001_1_01_000000_0
2016-05-17 21:29:45,505 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520585220_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:45,506 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FCFD7BBC133548FBADD722570DE14202, taskAttemptId=attempt_1463520585220_0001_1_01_000000_0, startTime=1463520585467, finishTime=1463520585506, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=143, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520585494, Process_Duration=8, Process_End_Time=1463520585502, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:45,506 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:45,506 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520585220_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:45,506 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520585220_0001_1_01_000000_0
2016-05-17 21:29:45,507 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:45,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:TASK_FINISHED]: vertexName=FCFD7BBC133548FBADD722570DE14202, taskId=task_1463520585220_0001_1_01_000000, startTime=1463520585467, finishTime=1463520585506, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1463520585220_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=143, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520585494, Process_Duration=8, Process_End_Time=1463520585502, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:45,507 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520585220_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:45,507 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202] : 1
2016-05-17 21:29:45,507 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202]
2016-05-17 21:29:45,507 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:45,508 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:45,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:VERTEX_FINISHED]: vertexName=FCFD7BBC133548FBADD722570DE14202, vertexId=vertex_1463520585220_0001_1_01, initRequestedTime=1463520585378, initedTime=1463520585383, startRequestedTime=1463520585390, startedTime=1463520585390, finishTime=1463520585507, timeTaken=117, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=143, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520585494, Process_Duration=8, Process_End_Time=1463520585502, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520585467, firstTasksToStart=[ task_1463520585220_0001_1_01_000000 ], lastTaskFinishTime=1463520585506, lastTasksToFinish=[ task_1463520585220_0001_1_01_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520585220_0001_1_01_000000 ], longestDurationTasks=[ task_1463520585220_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:45,508 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:45,508 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585220_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:45,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520585220_0001_00_000002, stoppedTime=1463520585508, exitStatus=0
2016-05-17 21:29:45,509 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585220_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:45,509 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:45,508 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:45,509 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520585220_0001_1
2016-05-17 21:29:45,509 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520585220_0001_1
2016-05-17 21:29:45,509 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520585220_0001_1_00 [ADA5E9343780468A8B98F9C6219D5D24]
2016-05-17 21:29:45,509 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: C54E2711C04648D1B1301EF1274F89F6 for vertex: vertex_1463520585220_0001_1_01 [FCFD7BBC133548FBADD722570DE14202]
2016-05-17 21:29:45,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=143, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520585494, Process_Duration=8, Process_End_Time=1463520585502, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:45,512 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:45,512 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:45,512 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520585220_0001_00_000002
2016-05-17 21:29:45,513 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520585220_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:45,513 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520585220_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:45,513 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520585220_0001_00_000002, Exiting
2016-05-17 21:29:45,513 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520585220_0001_00_000002
2016-05-17 21:29:45,513 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520585220_0001_00_000002
2016-05-17 21:29:45,513 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520585220_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:45,513 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520585220_0001_00_000002 completed successfully
2016-05-17 21:29:45,517 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520585220_0001_1, queueSize=0
2016-05-17 21:29:45,517 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:45,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585220_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520585220_0001_1, startTime=1463520585376, finishTime=1463520585516, timeTaken=140, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=876, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041170938, Process_Duration=10, Process_End_Time=2927041170948, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:45,518 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520585220_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:45,518 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520585220_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:45,518 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585220_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:45,518 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520585220_0001_00_000002
2016-05-17 21:29:45 Completed Dag: dag_1463520585220_0001_1
2016-05-17 21:29:45,518 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520585220_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:45,518 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520585220_0001_1_post
2016-05-17 21:29:45,518 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:45,518 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:45,524 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...emblyplatform/0_ARGS_SWAP, applicationId=application_1463520585220_0001
2016-05-17 21:29:45,525 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:45,525 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:45,525 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:45,526 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:45,526 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:45,526 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:45,526 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:45,526 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:45,526 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:45,527 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:45,527 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:45,527 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:45,527 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:45,531 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/0B2779ADBF1F496585B91CE8131D36F6/CC5243738A704BAAB076A2EDCEFAEDFA/.tez/application_1463520585220_0001
2016-05-17 21:29:45,532 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_SWAP/_temporary
2016-05-17 21:29:45,532 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.327
2016-05-17 21:29:45,593 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_SWAP
2016-05-17 21:29:45,612 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:45,613 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:45,645 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 21:29:45,648 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 21:29:45,648 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:45,648 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:45,649 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:45,670 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:45,674 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_-1
2016-05-17 21:29:45,714 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:45,714 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:45,714 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_-1"]
2016-05-17 21:29:45,714 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:45,715 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:45,715 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:45,715 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...lyplatform/ALL_'field'_-1
2016-05-17 21:29:45,715 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:45,726 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:45,730 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:45,731 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:45,731 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:45,740 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BC634643A9C34CA79B44C7D98CCEA66B/EB8CCFCBACA7452286956BDB0F31416B/.tez/application_1463520585730_0001 doesn't exist and is created
2016-05-17 21:29:45,754 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:45,754 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:45,754 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BC634643A9C34CA79B44C7D98CCEA66B/EB8CCFCBACA7452286956BDB0F31416B/.tez/application_1463520585730_0001
2016-05-17 21:29:45,757 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520585730_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:45,766 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:45,767 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:45,768 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:45,768 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:45,768 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BC634643A9C34CA79B44C7D98CCEA66B/EB8CCFCBACA7452286956BDB0F31416B/.tez/application_1463520585730_0001/localmode-log-dir
2016-05-17 21:29:45,769 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BC634643A9C34CA79B44C7D98CCEA66B/EB8CCFCBACA7452286956BDB0F31416B/.tez/application_1463520585730_0001/localmode-log-dir/history.txt.appattempt_1463520585730_0001_000000, maxErrors=10
2016-05-17 21:29:45,769 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:45,769 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520585730_0001_000000, appSubmitTime=1463520585756, launchTime=1463520585757
2016-05-17 21:29:45,774 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:52843
2016-05-17 21:29:45,776 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:45,778 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:45,780 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:45,780 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:45,781 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:45,783 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:45,783 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520585730_0001_000000, startTime=1463520585783
2016-05-17 21:29:45,783 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:45,854 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:45,854 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:45,854 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_'field'_-1, applicationId=application_1463520585730_0001, dagName=[BC634643A9C34CA79B44C7D98CCEA66B/EB8CCFCBACA7452286956BDB0F31416B] every/(1/1) ...lyplatform/ALL_'field'_-1
2016-05-17 21:29:45,872 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [BC634643A9C34CA79B44C7D98CCEA66B/EB8CCFCBACA7452286956BDB0F31416B] every/(1/1) ...lyplatform/ALL_'field'_-1
2016-05-17 21:29:45,874 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520585730_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BC634643A9C34CA79B44C7D98CCEA66B/EB8CCFCBACA7452286956BDB0F31416B/.tez/application_1463520585730_0001/localmode-log-dir/dag_1463520585730_0001_1.dot
2016-05-17 21:29:45,874 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BC634643A9C34CA79B44C7D98CCEA66B/EB8CCFCBACA7452286956BDB0F31416B/.tez/application_1463520585730_0001/localmode-log-dir/dag_1463520585730_0001_1-tez-dag.pb.txt
2016-05-17 21:29:45,882 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520585730_0001_1
2016-05-17 21:29:45,882 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:45,882 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [BC634643A9C34CA79B44C7D98CCEA66B/EB8CCFCBACA7452286956BDB0F31416B] every/(1/1) ...lyplatform/ALL_'field'_-1
2016-05-17 21:29:45 Running Dag: dag_1463520585730_0001_1
2016-05-17 21:29:45,895 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520585730_0001_1, submitTime=1463520585872
2016-05-17 21:29:45,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:45,896 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex ADB7D4CC506041BB9423778FA54A2742
2016-05-17 21:29:45,896 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:45,896 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520585730_0001_1, initTime=1463520585895
2016-05-17 21:29:45,896 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:45,896 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520585730_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:45,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:DAG_STARTED]: dagID=dag_1463520585730_0001_1, startTime=1463520585897
2016-05-17 21:29:45,897 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:45,897 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520585730_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:45,898 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B590652E1B2C4285A9D42627BAB9D55E : {12B20D547A814B58B676FC9723732A5B={InputName=12B20D547A814B58B676FC9723732A5B}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:45,898 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 12B20D547A814B58B676FC9723732A5B, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:45,898 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E]
2016-05-17 21:29:45,897 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:45,898 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E]
2016-05-17 21:29:45,897 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_'field'_-1, applicationId=application_1463520585730_0001, dagName=[BC634643A9C34CA79B44C7D98CCEA66B/EB8CCFCBACA7452286956BDB0F31416B] every/(1/1) ...lyplatform/ALL_'field'_-1
2016-05-17 21:29:45,898 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E]
2016-05-17 21:29:45,898 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E]. Starting root input initializers: 1
2016-05-17 21:29:45,899 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:45,899 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742]
2016-05-17 21:29:45,899 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 12B20D547A814B58B676FC9723732A5B on vertex vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E]
2016-05-17 21:29:45,901 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:45,901 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 12B20D547A814B58B676FC9723732A5B asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:45,901 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:45,901 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:45,902 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:45,902 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742]
2016-05-17 21:29:45,902 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742]
2016-05-17 21:29:45,903 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742]
2016-05-17 21:29:45,903 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=21DC368A54E54123A1729478AB8AD8C5, vertexId=vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:45,903 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=21DC368A54E54123A1729478AB8AD8C5, vertexId=vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742]
2016-05-17 21:29:45,905 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:45,906 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 12B20D547A814B58B676FC9723732A5B on vertex vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E]
2016-05-17 21:29:45,905 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=21DC368A54E54123A1729478AB8AD8C5, vertexName=ADB7D4CC506041BB9423778FA54A2742, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:45,906 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=21DC368A54E54123A1729478AB8AD8C5, vertexId=vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742]
2016-05-17 21:29:45,907 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:45,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ADB7D4CC506041BB9423778FA54A2742, vertexId=vertex_1463520585730_0001_1_01, initRequestedTime=1463520585899, initedTime=1463520585907, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:45,908 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:45,908 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:45,908 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {12B20D547A814B58B676FC9723732A5B=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:45,908 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E] parallelism set to 1
2016-05-17 21:29:45,908 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:45,909 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520585730_0001_1_00, eventCount=1
2016-05-17 21:29:45,909 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:45,910 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B590652E1B2C4285A9D42627BAB9D55E, vertexId=vertex_1463520585730_0001_1_00, initRequestedTime=1463520585898, initedTime=1463520585909, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:45,910 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E] with distanceFromRoot: 0
2016-05-17 21:29:45,910 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:45,910 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B590652E1B2C4285A9D42627BAB9D55E
2016-05-17 21:29:45,910 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E]
2016-05-17 21:29:45,911 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:45,911 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520585730_0001_1_00, startRequestedTime=1463520585910, startedTime=1463520585910
2016-05-17 21:29:45,911 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:45,912 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:45,912 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:TASK_STARTED]: vertexName=B590652E1B2C4285A9D42627BAB9D55E, taskId=task_1463520585730_0001_1_00_000000, scheduledTime=1463520585911, launchTime=1463520585911
2016-05-17 21:29:45,912 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520585730_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:45,912 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520585730_0001_1_00 for vertex: vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742] numStartedSources: 1 numSources: 1
2016-05-17 21:29:45,912 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742] with distanceFromRoot: 1
2016-05-17 21:29:45,912 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E]
2016-05-17 21:29:45,913 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B590652E1B2C4285A9D42627BAB9D55E in vertex: ADB7D4CC506041BB9423778FA54A2742
2016-05-17 21:29:45,913 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: ADB7D4CC506041BB9423778FA54A2742 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:45,913 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:45,914 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520585730_0001_1_01, startRequestedTime=1463520585913, startedTime=1463520585913
2016-05-17 21:29:45,914 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:45,914 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BC634643A9C34CA79B44C7D98CCEA66B/EB8CCFCBACA7452286956BDB0F31416B] every/(1/1) ...lyplatform/ALL_'field'_-1, VertexName: B590652E1B2C4285A9D42627BAB9D55E, VertexParallelism: 1, TaskAttemptID:attempt_1463520585730_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=12B20D547A814B58B676FC9723732A5B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=ADB7D4CC506041BB9423778FA54A2742, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:45,914 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520585730_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:45,914 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:45,915 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:45,915 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:45,915 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:45,916 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520585730_0001_00_000001
2016-05-17 21:29:45,916 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585730_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:45,917 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520585730_0001_00_000001
2016-05-17 21:29:45,917 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:45,918 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:45,918 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585730_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:45,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520585730_0001_00_000001, launchTime=1463520585918
2016-05-17 21:29:45,919 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520585730_0001_00_000001
2016-05-17 21:29:45,919 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520585730_0001_1_00_000000_0] to container: [container_1463520585730_0001_00_000001]
2016-05-17 21:29:45,919 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585730_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:45,919 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520585730_0001_00_000001 given task: attempt_1463520585730_0001_1_00_000000_0
2016-05-17 21:29:45,920 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520585730_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520585730_0001_1_00_000000_0
2016-05-17 21:29:45,920 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520585730_0001_1_00_000000_0] started. Is using containerId: [container_1463520585730_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:45 Starting to run new task attempt: attempt_1463520585730_0001_1_00_000000_0
2016-05-17 21:29:45,920 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:45,920 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520585730_0001_1_00_000000_0
2016-05-17 21:29:45,920 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:45,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B590652E1B2C4285A9D42627BAB9D55E, taskAttemptId=attempt_1463520585730_0001_1_00_000000_0, startTime=1463520585920, containerId=container_1463520585730_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520585730_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:45,920 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:45,920 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:45,920 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520585730_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:45,921 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:45,921 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520585730_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:45,921 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:45,921 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:45,921 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BC634643A9C34CA79B44C7D98CCEA66B/EB8CCFCBACA7452286956BDB0F31416B] every/(1/1) ...lyplatform/ALL_'field'_-1, VertexName: B590652E1B2C4285A9D42627BAB9D55E, VertexParallelism: 1, TaskAttemptID:attempt_1463520585730_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=12B20D547A814B58B676FC9723732A5B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=ADB7D4CC506041BB9423778FA54A2742, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:45,921 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:45,922 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520585730_0001_1_00_000000_0
2016-05-17 21:29:45,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:45,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:45,923 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520585730_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:45,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=12B20D547A814B58B676FC9723732A5B, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:45,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:45,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 12B20D547A814B58B676FC9723732A5B
2016-05-17 21:29:45,924 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 12B20D547A814B58B676FC9723732A5B
2016-05-17 21:29:45,925 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:45,926 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 12B20D547A814B58B676FC9723732A5B
2016-05-17 21:29:45,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 12B20D547A814B58B676FC9723732A5B
2016-05-17 21:29:45,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:45,927 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520585730_0001, with dag id: dag_1463520585730_0001_1
2016-05-17 21:29:45,928 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:45,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=ADB7D4CC506041BB9423778FA54A2742, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:45,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:45,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: ADB7D4CC506041BB9423778FA54A2742
2016-05-17 21:29:45,932 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B590652E1B2C4285A9D42627BAB9D55E, ordinal: 0
2016-05-17 21:29:45,933 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B590652E1B2C4285A9D42627BAB9D55E, mem on start (mb), free: 107, total: 364, max: 672
2016-05-17 21:29:45,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:45,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:45,933 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:45,934 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:45,934 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: ADB7D4CC506041BB9423778FA54A2742
2016-05-17 21:29:45,934 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: ADB7D4CC506041BB9423778FA54A2742
2016-05-17 21:29:45,934 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:45,934 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:45,935 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:45,935 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:45,935 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:45,935 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:45,935 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:45,936 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 12B20D547A814B58B676FC9723732A5B, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:45,936 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, ADB7D4CC506041BB9423778FA54A2742, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:45,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:45,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 12B20D547A814B58B676FC9723732A5B
2016-05-17 21:29:45,936 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:45,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 12B20D547A814B58B676FC9723732A5B
2016-05-17 21:29:45,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 12B20D547A814B58B676FC9723732A5B being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:45,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:45,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:45,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:45,937 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520585730_0001_1_00_000000_0
2016-05-17 21:29:45,939 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:45,939 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6ca5d944
2016-05-17 21:29:45,940 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:45,941 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 12B20D547A814B58B676FC9723732A5B
2016-05-17 21:29:45,941 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 4C3178DE50444E0A9F5F0897734E108A
2016-05-17 21:29:45,941 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 4C3178DE50444E0A9F5F0897734E108A
2016-05-17 21:29:45,941 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6ca5d944
2016-05-17 21:29:45,941 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:45,942 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:45,942 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:45,945 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:45,945 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@1cc6cca4; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@3fa531be; comparator=cascading.tuple.hadoop.util.TupleComparator@6f5c0b7b; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:45,946 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:45,946 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:45,946 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:45,957 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:45,957 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:45,970 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:45,970 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:45,970 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:45,970 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:45,970 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:45,971 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:45,971 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:45,971 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B590652E1B2C4285A9D42627BAB9D55E, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:45,974 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520585730_0001_1_00_000000_0
2016-05-17 21:29:45,974 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B590652E1B2C4285A9D42627BAB9D55E, mem on close (mb), free: 204, total: 364, max: 672
2016-05-17 21:29:45,974 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:45,974 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:45,974 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:45,974 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:45,977 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:45 Completed running task attempt: attempt_1463520585730_0001_1_00_000000_0
2016-05-17 21:29:45,979 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520585730_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:45,979 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520585971, Process_Duration=3, Process_End_Time=1463520585974, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:45,979 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520585730_0001_1_00, eventCount=1
2016-05-17 21:29:45,979 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:45,979 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:45,980 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520585730_0001_00_000001
2016-05-17 21:29:45,980 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B590652E1B2C4285A9D42627BAB9D55E, taskAttemptId=attempt_1463520585730_0001_1_00_000000_0, startTime=1463520585920, finishTime=1463520585979, timeTaken=59, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520585971, Process_Duration=3, Process_End_Time=1463520585974, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:45,980 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:45,980 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520585730_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:45,980 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520585730_0001_1_00_000000_0
2016-05-17 21:29:45,980 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:45,981 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:TASK_FINISHED]: vertexName=B590652E1B2C4285A9D42627BAB9D55E, taskId=task_1463520585730_0001_1_00_000000, startTime=1463520585920, finishTime=1463520585980, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1463520585730_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520585971, Process_Duration=3, Process_End_Time=1463520585974, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:45,981 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:45,981 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520585730_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:45,981 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E] : 1
2016-05-17 21:29:45,982 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E]
2016-05-17 21:29:45,982 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:45,982 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:VERTEX_FINISHED]: vertexName=B590652E1B2C4285A9D42627BAB9D55E, vertexId=vertex_1463520585730_0001_1_00, initRequestedTime=1463520585898, initedTime=1463520585909, startRequestedTime=1463520585910, startedTime=1463520585910, finishTime=1463520585982, timeTaken=72, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520585971, Process_Duration=3, Process_End_Time=1463520585974, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520585920, firstTasksToStart=[ task_1463520585730_0001_1_00_000000 ], lastTaskFinishTime=1463520585979, lastTasksToFinish=[ task_1463520585730_0001_1_00_000000 ], minTaskDuration=59, maxTaskDuration=59, avgTaskDuration=59.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520585730_0001_1_00_000000 ], longestDurationTasks=[ task_1463520585730_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:45,982 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:45,982 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585730_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:45,982 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520585730_0001_00_000001, stoppedTime=1463520585982, exitStatus=0
2016-05-17 21:29:45,983 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585730_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:45,983 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742] attempt: attempt_1463520585730_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:45,983 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: ADB7D4CC506041BB9423778FA54A2742
2016-05-17 21:29:45,983 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742]
2016-05-17 21:29:45,983 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:45,983 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:45,983 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520585730_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:45,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:TASK_STARTED]: vertexName=ADB7D4CC506041BB9423778FA54A2742, taskId=task_1463520585730_0001_1_01_000000, scheduledTime=1463520585983, launchTime=1463520585983
2016-05-17 21:29:45,984 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520585730_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:45,984 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742]
2016-05-17 21:29:45,984 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585730_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:45,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:45,984 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [BC634643A9C34CA79B44C7D98CCEA66B/EB8CCFCBACA7452286956BDB0F31416B] every/(1/1) ...lyplatform/ALL_'field'_-1, VertexName: ADB7D4CC506041BB9423778FA54A2742, VertexParallelism: 1, TaskAttemptID:attempt_1463520585730_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B590652E1B2C4285A9D42627BAB9D55E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=21DC368A54E54123A1729478AB8AD8C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:45,984 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520585730_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:45,984 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:45,985 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520585730_0001_00_000002
2016-05-17 21:29:45,985 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585730_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:45,987 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520585730_0001_00_000002
2016-05-17 21:29:45,987 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:45,987 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585730_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:45,988 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:45,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520585730_0001_00_000002, launchTime=1463520585987
2016-05-17 21:29:45,988 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520585730_0001_00_000002
2016-05-17 21:29:45,988 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520585730_0001_1_01_000000_0] to container: [container_1463520585730_0001_00_000002]
2016-05-17 21:29:45,988 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585730_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:45,989 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520585730_0001_00_000002 given task: attempt_1463520585730_0001_1_01_000000_0
2016-05-17 21:29:45,989 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520585730_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520585730_0001_1_01_000000_0
2016-05-17 21:29:45,989 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520585730_0001_1_01_000000_0] started. Is using containerId: [container_1463520585730_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:45,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ADB7D4CC506041BB9423778FA54A2742, taskAttemptId=attempt_1463520585730_0001_1_01_000000_0, startTime=1463520585989, containerId=container_1463520585730_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520585730_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:45 Starting to run new task attempt: attempt_1463520585730_0001_1_01_000000_0
2016-05-17 21:29:45,989 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:45,989 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520585730_0001_1_01_000000_0
2016-05-17 21:29:45,990 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:45,989 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520585730_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:45,990 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:45,990 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520585730_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:45,990 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520585730_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:45,990 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520585730_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:45,990 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520585730_0001_00_000001, Exiting
2016-05-17 21:29:45,990 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520585730_0001_00_000001
2016-05-17 21:29:45,990 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520585730_0001_00_000001
2016-05-17 21:29:45,990 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:45,991 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:45,991 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:45,991 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:45,991 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [BC634643A9C34CA79B44C7D98CCEA66B/EB8CCFCBACA7452286956BDB0F31416B] every/(1/1) ...lyplatform/ALL_'field'_-1, VertexName: ADB7D4CC506041BB9423778FA54A2742, VertexParallelism: 1, TaskAttemptID:attempt_1463520585730_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B590652E1B2C4285A9D42627BAB9D55E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=21DC368A54E54123A1729478AB8AD8C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:45,991 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520585730_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:45,991 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:45,991 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520585730_0001_00_000001 completed successfully
2016-05-17 21:29:45,992 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585730_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:45,992 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520585730_0001_1_01_000000_0
2016-05-17 21:29:45,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:45,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:45,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:45,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B590652E1B2C4285A9D42627BAB9D55E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:45,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:45,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B590652E1B2C4285A9D42627BAB9D55E
2016-05-17 21:29:45,995 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:45,996 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:45,996 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:45,996 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B590652E1B2C4285A9D42627BAB9D55E
2016-05-17 21:29:45,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B590652E1B2C4285A9D42627BAB9D55E
2016-05-17 21:29:45,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=21DC368A54E54123A1729478AB8AD8C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:45,997 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520585730_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:45,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:45,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 21DC368A54E54123A1729478AB8AD8C5
2016-05-17 21:29:45,998 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:45,998 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 21DC368A54E54123A1729478AB8AD8C5
2016-05-17 21:29:45,998 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ADB7D4CC506041BB9423778FA54A2742, ordinal: 1
2016-05-17 21:29:45,999 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADB7D4CC506041BB9423778FA54A2742, mem on start (mb), free: 199, total: 364, max: 672
2016-05-17 21:29:45,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:45,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:45,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:46,005 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:46,006 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 21DC368A54E54123A1729478AB8AD8C5
2016-05-17 21:29:46,006 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:46,006 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:46,006 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:46,006 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:46,006 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:46,007 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:46,007 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:46,007 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B590652E1B2C4285A9D42627BAB9D55E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:46,007 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 21DC368A54E54123A1729478AB8AD8C5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:46,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:46,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B590652E1B2C4285A9D42627BAB9D55E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:46,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:46,008 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:46,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B590652E1B2C4285A9D42627BAB9D55E
2016-05-17 21:29:46,008 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:46,009 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:46,009 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B590652E1B2C4285A9D42627BAB9D55E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:46,009 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:46,009 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:46,009 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:46,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B590652E1B2C4285A9D42627BAB9D55E
2016-05-17 21:29:46,011 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:46,012 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520585730_0001_1_01_000000_0
2016-05-17 21:29:46,012 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520585730_0001_1_00_000000_0_10231, runDuration: 46598, ]
2016-05-17 21:29:46,015 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:46,016 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 4C3178DE50444E0A9F5F0897734E108A
2016-05-17 21:29:46,016 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_-1"], id: 21DC368A54E54123A1729478AB8AD8C5
2016-05-17 21:29:46,016 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_-1"]
2016-05-17 21:29:46,016 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 4C3178DE50444E0A9F5F0897734E108A, for 1 inputs
2016-05-17 21:29:46,018 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520585730_0001_1_00_000000_0_10231] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520586018, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:46,018 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B590652E1B2C4285A9D42627BAB9D55E
2016-05-17 21:29:46,018 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:46,018 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B590652E1B2C4285A9D42627BAB9D55E] #39 in 1ms
2016-05-17 21:29:46,020 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B590652E1B2C4285A9D42627BAB9D55E] #39, status:false, isInterrupted:false
2016-05-17 21:29:46,021 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:46,022 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:46,022 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:46,023 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:46,023 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:46,023 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:46,024 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B590652E1B2C4285A9D42627BAB9D55E
2016-05-17 21:29:46,024 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ADB7D4CC506041BB9423778FA54A2742, all 1 inputs ready in: 00:00:00.008
2016-05-17 21:29:46,024 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:46,026 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:46,027 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@42ee3073; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@23aef059
2016-05-17 21:29:46,030 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:46,030 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:46,030 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520585730_0001_1_01_000000_0
2016-05-17 21:29:46,030 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520585730_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:46,031 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520585730_0001_1_01_000000_0
2016-05-17 21:29:46,031 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADB7D4CC506041BB9423778FA54A2742, mem on close (mb), free: 195, total: 364, max: 672
2016-05-17 21:29:46,031 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B590652E1B2C4285A9D42627BAB9D55E
2016-05-17 21:29:46,032 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:46,032 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:46,033 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:46,033 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:46,033 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:46,033 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:46 Completed running task attempt: attempt_1463520585730_0001_1_01_000000_0
2016-05-17 21:29:46,034 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520585730_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:46,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520586024, Process_Duration=7, Process_End_Time=1463520586031, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:46,035 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ADB7D4CC506041BB9423778FA54A2742, taskAttemptId=attempt_1463520585730_0001_1_01_000000_0, startTime=1463520585989, finishTime=1463520586034, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520586024, Process_Duration=7, Process_End_Time=1463520586031, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:46,035 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:46,035 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:46,035 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520585730_0001_00_000002
2016-05-17 21:29:46,035 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:46,035 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520585730_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:46,036 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520585730_0001_1_01_000000_0
2016-05-17 21:29:46,036 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:46,036 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:46,036 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:TASK_FINISHED]: vertexName=ADB7D4CC506041BB9423778FA54A2742, taskId=task_1463520585730_0001_1_01_000000, startTime=1463520585989, finishTime=1463520586036, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1463520585730_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520586024, Process_Duration=7, Process_End_Time=1463520586031, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:46,037 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520585730_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:46,037 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742] : 1
2016-05-17 21:29:46,037 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742]
2016-05-17 21:29:46,037 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:46,037 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:VERTEX_FINISHED]: vertexName=ADB7D4CC506041BB9423778FA54A2742, vertexId=vertex_1463520585730_0001_1_01, initRequestedTime=1463520585899, initedTime=1463520585907, startRequestedTime=1463520585913, startedTime=1463520585913, finishTime=1463520586037, timeTaken=124, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520586024, Process_Duration=7, Process_End_Time=1463520586031, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520585989, firstTasksToStart=[ task_1463520585730_0001_1_01_000000 ], lastTaskFinishTime=1463520586034, lastTasksToFinish=[ task_1463520585730_0001_1_01_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520585730_0001_1_01_000000 ], longestDurationTasks=[ task_1463520585730_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:46,037 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:46,037 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585730_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:46,038 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520585730_0001_00_000002, stoppedTime=1463520586038, exitStatus=0
2016-05-17 21:29:46,038 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:46,038 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585730_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:46,038 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:46,038 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520585730_0001_1
2016-05-17 21:29:46,038 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520585730_0001_1
2016-05-17 21:29:46,038 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520585730_0001_1_00 [B590652E1B2C4285A9D42627BAB9D55E]
2016-05-17 21:29:46,038 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 21DC368A54E54123A1729478AB8AD8C5 for vertex: vertex_1463520585730_0001_1_01 [ADB7D4CC506041BB9423778FA54A2742]
2016-05-17 21:29:46,045 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520585730_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:46,046 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520585730_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:46,046 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520585730_0001_00_000002, Exiting
2016-05-17 21:29:46,046 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520585730_0001_00_000002
2016-05-17 21:29:46,046 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520585730_0001_00_000002
2016-05-17 21:29:46,046 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520585730_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:46,046 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520585730_0001_00_000002 completed successfully
2016-05-17 21:29:46,046 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520585730_0001_1, queueSize=0
2016-05-17 21:29:46,047 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520585730_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520585730_0001_1, startTime=1463520585897, finishTime=1463520586045, timeTaken=148, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=345, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041171995, Process_Duration=10, Process_End_Time=2927041172005, Read_Duration=5, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:46,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:46,047 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520585730_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:46,047 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520585730_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:46,048 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520585730_0001_00_000002
2016-05-17 21:29:46,048 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520585730_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:46 Completed Dag: dag_1463520585730_0001_1
2016-05-17 21:29:46,049 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520585730_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:46,050 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520585730_0001_1_post
2016-05-17 21:29:46,051 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:46,051 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:46,056 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/ALL_'field'_-1, applicationId=application_1463520585730_0001
2016-05-17 21:29:46,058 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:46,058 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:46,058 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:46,058 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:46,058 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:46,059 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:46,059 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:46,059 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:46,059 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:46,059 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:46,060 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:46,060 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:46,061 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:46,065 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/BC634643A9C34CA79B44C7D98CCEA66B/EB8CCFCBACA7452286956BDB0F31416B/.tez/application_1463520585730_0001
2016-05-17 21:29:46,066 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_-1/_temporary
2016-05-17 21:29:46,066 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.351
2016-05-17 21:29:46,128 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_-1
2016-05-17 21:29:46,146 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:46,146 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:46,178 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 21:29:46,179 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-05-17 21:29:46,179 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:46,179 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:46,180 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:46,194 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:46,199 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_'field'
2016-05-17 21:29:46,241 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:46,241 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:46,241 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_'field'"]
2016-05-17 21:29:46,241 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:46,242 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:46,242 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:46,242 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...tform/ALL_'field'_'field'
2016-05-17 21:29:46,242 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:46,252 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:46,257 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:46,257 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:46,257 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:46,266 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/24819D7C1AF24D4185DEADC4C5B4C490/BEA2C00810AA4A248E31716D4DDF0D52/.tez/application_1463520586257_0001 doesn't exist and is created
2016-05-17 21:29:46,280 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:46,280 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:46,281 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/24819D7C1AF24D4185DEADC4C5B4C490/BEA2C00810AA4A248E31716D4DDF0D52/.tez/application_1463520586257_0001
2016-05-17 21:29:46,282 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520586257_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:46,291 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:46,292 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:46,292 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:46,293 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:46,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/24819D7C1AF24D4185DEADC4C5B4C490/BEA2C00810AA4A248E31716D4DDF0D52/.tez/application_1463520586257_0001/localmode-log-dir
2016-05-17 21:29:46,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/24819D7C1AF24D4185DEADC4C5B4C490/BEA2C00810AA4A248E31716D4DDF0D52/.tez/application_1463520586257_0001/localmode-log-dir/history.txt.appattempt_1463520586257_0001_000000, maxErrors=10
2016-05-17 21:29:46,293 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:46,293 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520586257_0001_000000, appSubmitTime=1463520586282, launchTime=1463520586282
2016-05-17 21:29:46,297 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:46,301 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:46,302 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:46,305 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:46,306 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:48889
2016-05-17 21:29:46,308 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:46,310 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520586257_0001_000000, startTime=1463520586310
2016-05-17 21:29:46,310 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:46,310 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:46,380 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:46,380 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:46,381 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_'field', applicationId=application_1463520586257_0001, dagName=[24819D7C1AF24D4185DEADC4C5B4C490/BEA2C00810AA4A248E31716D4DDF0D52] every/(1/1) ...tform/ALL_'field'_'field'
2016-05-17 21:29:46,396 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [24819D7C1AF24D4185DEADC4C5B4C490/BEA2C00810AA4A248E31716D4DDF0D52] every/(1/1) ...tform/ALL_'field'_'field'
2016-05-17 21:29:46,397 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520586257_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/24819D7C1AF24D4185DEADC4C5B4C490/BEA2C00810AA4A248E31716D4DDF0D52/.tez/application_1463520586257_0001/localmode-log-dir/dag_1463520586257_0001_1.dot
2016-05-17 21:29:46,397 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/24819D7C1AF24D4185DEADC4C5B4C490/BEA2C00810AA4A248E31716D4DDF0D52/.tez/application_1463520586257_0001/localmode-log-dir/dag_1463520586257_0001_1-tez-dag.pb.txt
2016-05-17 21:29:46,402 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520586257_0001_1
2016-05-17 21:29:46,402 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:46,402 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [24819D7C1AF24D4185DEADC4C5B4C490/BEA2C00810AA4A248E31716D4DDF0D52] every/(1/1) ...tform/ALL_'field'_'field'
2016-05-17 21:29:46 Running Dag: dag_1463520586257_0001_1
2016-05-17 21:29:46,413 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520586257_0001_1, submitTime=1463520586396
2016-05-17 21:29:46,413 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:46,413 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D5DA12156D9A499B9C11173561BC637D
2016-05-17 21:29:46,413 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:46,413 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:46,414 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520586257_0001_1, initTime=1463520586413
2016-05-17 21:29:46,414 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520586257_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:46,414 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:DAG_STARTED]: dagID=dag_1463520586257_0001_1, startTime=1463520586414
2016-05-17 21:29:46,414 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:46,414 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520586257_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:46,414 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C938E68BD4F043F78126DDCD2033CB18 : {C605E99132EF4D98A54D6D23325202D6={InputName=C605E99132EF4D98A54D6D23325202D6}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:46,414 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C605E99132EF4D98A54D6D23325202D6, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:46,415 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18]
2016-05-17 21:29:46,415 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18]
2016-05-17 21:29:46,415 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18]
2016-05-17 21:29:46,415 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18]. Starting root input initializers: 1
2016-05-17 21:29:46,415 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:46,415 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:46,415 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D]
2016-05-17 21:29:46,416 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_'field', applicationId=application_1463520586257_0001, dagName=[24819D7C1AF24D4185DEADC4C5B4C490/BEA2C00810AA4A248E31716D4DDF0D52] every/(1/1) ...tform/ALL_'field'_'field'
2016-05-17 21:29:46,418 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C605E99132EF4D98A54D6D23325202D6 on vertex vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18]
2016-05-17 21:29:46,418 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:46,419 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D]
2016-05-17 21:29:46,419 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D]
2016-05-17 21:29:46,419 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D]
2016-05-17 21:29:46,419 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=41CD43C99F6D46A18826E7888235ECB1, vertexId=vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:46,419 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=41CD43C99F6D46A18826E7888235ECB1, vertexId=vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D]
2016-05-17 21:29:46,421 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=41CD43C99F6D46A18826E7888235ECB1, vertexName=D5DA12156D9A499B9C11173561BC637D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:46,421 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=41CD43C99F6D46A18826E7888235ECB1, vertexId=vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D]
2016-05-17 21:29:46,421 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:46,422 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C605E99132EF4D98A54D6D23325202D6 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:46,422 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:46,422 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:46,422 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:46,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D5DA12156D9A499B9C11173561BC637D, vertexId=vertex_1463520586257_0001_1_01, initRequestedTime=1463520586415, initedTime=1463520586422, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:46,423 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:46,423 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:46,424 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C605E99132EF4D98A54D6D23325202D6 on vertex vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18]
2016-05-17 21:29:46,424 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:46,424 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C605E99132EF4D98A54D6D23325202D6=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:46,424 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18] parallelism set to 1
2016-05-17 21:29:46,424 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:46,425 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520586257_0001_1_00, eventCount=1
2016-05-17 21:29:46,425 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C938E68BD4F043F78126DDCD2033CB18, vertexId=vertex_1463520586257_0001_1_00, initRequestedTime=1463520586414, initedTime=1463520586425, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:46,425 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18] with distanceFromRoot: 0
2016-05-17 21:29:46,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:46,425 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:46,425 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C938E68BD4F043F78126DDCD2033CB18
2016-05-17 21:29:46,426 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18]
2016-05-17 21:29:46,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520586257_0001_1_00, startRequestedTime=1463520586425, startedTime=1463520586425
2016-05-17 21:29:46,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:46,426 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:46,427 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:46,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:TASK_STARTED]: vertexName=C938E68BD4F043F78126DDCD2033CB18, taskId=task_1463520586257_0001_1_00_000000, scheduledTime=1463520586427, launchTime=1463520586427
2016-05-17 21:29:46,427 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520586257_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:46,427 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520586257_0001_1_00 for vertex: vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D] numStartedSources: 1 numSources: 1
2016-05-17 21:29:46,427 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D] with distanceFromRoot: 1
2016-05-17 21:29:46,427 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18]
2016-05-17 21:29:46,428 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C938E68BD4F043F78126DDCD2033CB18 in vertex: D5DA12156D9A499B9C11173561BC637D
2016-05-17 21:29:46,428 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D5DA12156D9A499B9C11173561BC637D with 1 source tasks and 1 pending tasks
2016-05-17 21:29:46,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520586257_0001_1_01, startRequestedTime=1463520586428, startedTime=1463520586428
2016-05-17 21:29:46,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:46,429 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:46,429 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [24819D7C1AF24D4185DEADC4C5B4C490/BEA2C00810AA4A248E31716D4DDF0D52] every/(1/1) ...tform/ALL_'field'_'field', VertexName: C938E68BD4F043F78126DDCD2033CB18, VertexParallelism: 1, TaskAttemptID:attempt_1463520586257_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C605E99132EF4D98A54D6D23325202D6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D5DA12156D9A499B9C11173561BC637D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:46,430 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520586257_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:46,430 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:46,430 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:46,431 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:46,431 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:46,431 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520586257_0001_00_000001
2016-05-17 21:29:46,431 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586257_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:46,432 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520586257_0001_00_000001
2016-05-17 21:29:46,432 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:46,433 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520586257_0001, with dag id: dag_1463520586257_0001_1
2016-05-17 21:29:46,433 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586257_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:46,433 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:46,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520586257_0001_00_000001, launchTime=1463520586433
2016-05-17 21:29:46,435 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520586257_0001_00_000001
2016-05-17 21:29:46,435 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520586257_0001_1_00_000000_0] to container: [container_1463520586257_0001_00_000001]
2016-05-17 21:29:46,435 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586257_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:46,436 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520586257_0001_00_000001 given task: attempt_1463520586257_0001_1_00_000000_0
2016-05-17 21:29:46,436 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520586257_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520586257_0001_1_00_000000_0
2016-05-17 21:29:46,436 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520586257_0001_1_00_000000_0] started. Is using containerId: [container_1463520586257_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:46,436 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C938E68BD4F043F78126DDCD2033CB18, taskAttemptId=attempt_1463520586257_0001_1_00_000000_0, startTime=1463520586436, containerId=container_1463520586257_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520586257_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:46,436 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520586257_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:46,436 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520586257_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:46,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:46 Starting to run new task attempt: attempt_1463520586257_0001_1_00_000000_0
2016-05-17 21:29:46,438 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520586257_0001_1_00_000000_0
2016-05-17 21:29:46,438 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:46,438 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:46,438 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:46,438 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:46,438 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:46,438 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:46,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [24819D7C1AF24D4185DEADC4C5B4C490/BEA2C00810AA4A248E31716D4DDF0D52] every/(1/1) ...tform/ALL_'field'_'field', VertexName: C938E68BD4F043F78126DDCD2033CB18, VertexParallelism: 1, TaskAttemptID:attempt_1463520586257_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C605E99132EF4D98A54D6D23325202D6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D5DA12156D9A499B9C11173561BC637D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:46,439 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:46,439 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520586257_0001_1_00_000000_0
2016-05-17 21:29:46,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:46,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:46,440 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520586257_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:46,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C605E99132EF4D98A54D6D23325202D6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:46,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:46,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D5DA12156D9A499B9C11173561BC637D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:46,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:46,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D5DA12156D9A499B9C11173561BC637D
2016-05-17 21:29:46,443 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:46,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:46,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C605E99132EF4D98A54D6D23325202D6
2016-05-17 21:29:46,443 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C605E99132EF4D98A54D6D23325202D6
2016-05-17 21:29:46,444 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:46,444 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D5DA12156D9A499B9C11173561BC637D
2016-05-17 21:29:46,444 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D5DA12156D9A499B9C11173561BC637D
2016-05-17 21:29:46,446 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:46,446 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C605E99132EF4D98A54D6D23325202D6
2016-05-17 21:29:46,446 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C605E99132EF4D98A54D6D23325202D6
2016-05-17 21:29:46,447 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C938E68BD4F043F78126DDCD2033CB18, ordinal: 0
2016-05-17 21:29:46,447 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C938E68BD4F043F78126DDCD2033CB18, mem on start (mb), free: 105, total: 364, max: 672
2016-05-17 21:29:46,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:46,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:46,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:46,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:46,447 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:46,448 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:46,448 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:46,448 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:46,448 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:46,449 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:46,450 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C605E99132EF4D98A54D6D23325202D6, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:46,450 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D5DA12156D9A499B9C11173561BC637D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:46,450 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:46,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C605E99132EF4D98A54D6D23325202D6 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:46,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:46,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:46,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C605E99132EF4D98A54D6D23325202D6
2016-05-17 21:29:46,451 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:46,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C605E99132EF4D98A54D6D23325202D6
2016-05-17 21:29:46,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:46,452 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520586257_0001_1_00_000000_0
2016-05-17 21:29:46,454 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:46,455 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C605E99132EF4D98A54D6D23325202D6
2016-05-17 21:29:46,455 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: A01FA10774614DD482F276A0555EC1CA
2016-05-17 21:29:46,455 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] A01FA10774614DD482F276A0555EC1CA
2016-05-17 21:29:46,456 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:46,457 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:46,457 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@9625154
2016-05-17 21:29:46,458 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:46,458 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@439592ae; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@10d1002c; comparator=cascading.tuple.hadoop.util.TupleComparator@15c9669c; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:46,458 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:46,458 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:46,459 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:46,468 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@9625154
2016-05-17 21:29:46,468 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:46,468 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:46,477 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:46,477 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:46,482 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:46,483 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:46,483 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:46,483 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:46,483 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:46,484 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:46,484 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:46,485 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C938E68BD4F043F78126DDCD2033CB18, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:46,487 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520586257_0001_1_00_000000_0
2016-05-17 21:29:46,488 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C938E68BD4F043F78126DDCD2033CB18, mem on close (mb), free: 203, total: 364, max: 672
2016-05-17 21:29:46,488 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:46,488 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:46,488 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:46,489 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:46,492 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:46 Completed running task attempt: attempt_1463520586257_0001_1_00_000000_0
2016-05-17 21:29:46,496 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520586257_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:46,496 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520586257_0001_1_00, eventCount=1
2016-05-17 21:29:46,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C938E68BD4F043F78126DDCD2033CB18, taskAttemptId=attempt_1463520586257_0001_1_00_000000_0, startTime=1463520586436, finishTime=1463520586497, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520586485, Process_Duration=2, Process_End_Time=1463520586487, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:46,497 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520586257_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:46,497 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520586257_0001_1_00_000000_0
2016-05-17 21:29:46,497 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:46,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:46,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:TASK_FINISHED]: vertexName=C938E68BD4F043F78126DDCD2033CB18, taskId=task_1463520586257_0001_1_00_000000, startTime=1463520586436, finishTime=1463520586497, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1463520586257_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520586485, Process_Duration=2, Process_End_Time=1463520586487, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:46,498 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520586257_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:46,498 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18] : 1
2016-05-17 21:29:46,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520586485, Process_Duration=2, Process_End_Time=1463520586487, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:46,499 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18]
2016-05-17 21:29:46,499 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:46,499 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:VERTEX_FINISHED]: vertexName=C938E68BD4F043F78126DDCD2033CB18, vertexId=vertex_1463520586257_0001_1_00, initRequestedTime=1463520586414, initedTime=1463520586425, startRequestedTime=1463520586425, startedTime=1463520586425, finishTime=1463520586499, timeTaken=74, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520586485, Process_Duration=2, Process_End_Time=1463520586487, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520586436, firstTasksToStart=[ task_1463520586257_0001_1_00_000000 ], lastTaskFinishTime=1463520586497, lastTasksToFinish=[ task_1463520586257_0001_1_00_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520586257_0001_1_00_000000 ], longestDurationTasks=[ task_1463520586257_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:46,499 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:46,499 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:46,499 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586257_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:46,499 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520586257_0001_00_000001, stoppedTime=1463520586499, exitStatus=0
2016-05-17 21:29:46,500 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586257_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:46,500 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:46,500 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D] attempt: attempt_1463520586257_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:46,500 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D5DA12156D9A499B9C11173561BC637D
2016-05-17 21:29:46,500 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D]
2016-05-17 21:29:46,500 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:46,500 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:46,500 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:46,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:TASK_STARTED]: vertexName=D5DA12156D9A499B9C11173561BC637D, taskId=task_1463520586257_0001_1_01_000000, scheduledTime=1463520586500, launchTime=1463520586500
2016-05-17 21:29:46,501 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520586257_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:46,500 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:46,501 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520586257_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:46,501 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D]
2016-05-17 21:29:46,501 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586257_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:46,501 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [24819D7C1AF24D4185DEADC4C5B4C490/BEA2C00810AA4A248E31716D4DDF0D52] every/(1/1) ...tform/ALL_'field'_'field', VertexName: D5DA12156D9A499B9C11173561BC637D, VertexParallelism: 1, TaskAttemptID:attempt_1463520586257_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C938E68BD4F043F78126DDCD2033CB18, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=41CD43C99F6D46A18826E7888235ECB1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:46,501 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520586257_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:46,501 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:46,502 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520586257_0001_00_000002
2016-05-17 21:29:46,502 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586257_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:46,502 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520586257_0001_00_000001
2016-05-17 21:29:46,502 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520586257_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:46,503 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520586257_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:46,504 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520586257_0001_00_000001, Exiting
2016-05-17 21:29:46,504 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520586257_0001_00_000001
2016-05-17 21:29:46,504 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520586257_0001_00_000001
2016-05-17 21:29:46,504 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520586257_0001_00_000002
2016-05-17 21:29:46,504 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520586257_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:46,505 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520586257_0001_00_000001 completed successfully
2016-05-17 21:29:46,505 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:46,505 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586257_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:46,507 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:46,507 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586257_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:46,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520586257_0001_00_000002, launchTime=1463520586507
2016-05-17 21:29:46,509 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520586257_0001_00_000002
2016-05-17 21:29:46,509 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520586257_0001_1_01_000000_0] to container: [container_1463520586257_0001_00_000002]
2016-05-17 21:29:46,509 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586257_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:46,509 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520586257_0001_1_01_000000_0] started. Is using containerId: [container_1463520586257_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:46,510 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D5DA12156D9A499B9C11173561BC637D, taskAttemptId=attempt_1463520586257_0001_1_01_000000_0, startTime=1463520586509, containerId=container_1463520586257_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520586257_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:46,510 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520586257_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:46,510 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:46,510 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520586257_0001_00_000002 given task: attempt_1463520586257_0001_1_01_000000_0
2016-05-17 21:29:46,510 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520586257_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:46,510 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520586257_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520586257_0001_1_01_000000_0
2016-05-17 21:29:46 Starting to run new task attempt: attempt_1463520586257_0001_1_01_000000_0
2016-05-17 21:29:46,511 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520586257_0001_1_01_000000_0
2016-05-17 21:29:46,511 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:46,511 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:46,511 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:46,511 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:46,511 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:46,511 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:46,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [24819D7C1AF24D4185DEADC4C5B4C490/BEA2C00810AA4A248E31716D4DDF0D52] every/(1/1) ...tform/ALL_'field'_'field', VertexName: D5DA12156D9A499B9C11173561BC637D, VertexParallelism: 1, TaskAttemptID:attempt_1463520586257_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C938E68BD4F043F78126DDCD2033CB18, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=41CD43C99F6D46A18826E7888235ECB1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:46,512 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:46,512 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520586257_0001_1_01_000000_0
2016-05-17 21:29:46,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:46,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:46,513 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520586257_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:46,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:46,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=41CD43C99F6D46A18826E7888235ECB1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:46,514 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:46,514 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 41CD43C99F6D46A18826E7888235ECB1
2016-05-17 21:29:46,514 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:46,514 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 41CD43C99F6D46A18826E7888235ECB1
2016-05-17 21:29:46,514 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C938E68BD4F043F78126DDCD2033CB18, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:46,515 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:46,515 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C938E68BD4F043F78126DDCD2033CB18
2016-05-17 21:29:46,515 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:46,517 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:46,517 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:46,517 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C938E68BD4F043F78126DDCD2033CB18
2016-05-17 21:29:46,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C938E68BD4F043F78126DDCD2033CB18
2016-05-17 21:29:46,521 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D5DA12156D9A499B9C11173561BC637D, ordinal: 1
2016-05-17 21:29:46,521 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5DA12156D9A499B9C11173561BC637D, mem on start (mb), free: 197, total: 364, max: 672
2016-05-17 21:29:46,521 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:46,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:46,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:46,523 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:46,523 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 41CD43C99F6D46A18826E7888235ECB1
2016-05-17 21:29:46,523 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:46,523 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:46,523 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:46,523 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:46,523 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:46,524 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:46,524 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:46,524 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 41CD43C99F6D46A18826E7888235ECB1, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:46,524 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C938E68BD4F043F78126DDCD2033CB18, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:46,524 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:46,524 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C938E68BD4F043F78126DDCD2033CB18 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:46,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C938E68BD4F043F78126DDCD2033CB18
2016-05-17 21:29:46,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:46,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:46,526 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:46,526 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:46,526 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C938E68BD4F043F78126DDCD2033CB18 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:46,526 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:46,526 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:46,526 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:46,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C938E68BD4F043F78126DDCD2033CB18
2016-05-17 21:29:46,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:46,527 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520586257_0001_1_01_000000_0
2016-05-17 21:29:46,530 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520586257_0001_1_00_000000_0_10236, runDuration: 51987, ]
2016-05-17 21:29:46,533 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:46,534 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: A01FA10774614DD482F276A0555EC1CA
2016-05-17 21:29:46,534 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_'field'"], id: 41CD43C99F6D46A18826E7888235ECB1
2016-05-17 21:29:46,534 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_'field'"]
2016-05-17 21:29:46,534 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] A01FA10774614DD482F276A0555EC1CA, for 1 inputs
2016-05-17 21:29:46,535 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520586257_0001_1_00_000000_0_10236] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520586535, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:46,535 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C938E68BD4F043F78126DDCD2033CB18
2016-05-17 21:29:46,535 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:46,535 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [C938E68BD4F043F78126DDCD2033CB18] #40 in 1ms
2016-05-17 21:29:46,536 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C938E68BD4F043F78126DDCD2033CB18] #40, status:false, isInterrupted:false
2016-05-17 21:29:46,537 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:46,538 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:46,538 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:46,539 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:46,539 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:46,539 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:46,540 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D5DA12156D9A499B9C11173561BC637D, all 1 inputs ready in: 00:00:00.006
2016-05-17 21:29:46,540 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C938E68BD4F043F78126DDCD2033CB18
2016-05-17 21:29:46,541 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:46,541 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:46,542 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@30d9a951; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@21fb1de5
2016-05-17 21:29:46,545 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:46,545 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:46,545 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520586257_0001_1_01_000000_0
2016-05-17 21:29:46,545 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520586257_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:46,546 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520586257_0001_1_01_000000_0
2016-05-17 21:29:46,546 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5DA12156D9A499B9C11173561BC637D, mem on close (mb), free: 194, total: 364, max: 672
2016-05-17 21:29:46,546 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C938E68BD4F043F78126DDCD2033CB18
2016-05-17 21:29:46,546 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:46,547 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:46,547 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:46,547 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:46,547 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:46,547 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:46 Completed running task attempt: attempt_1463520586257_0001_1_01_000000_0
2016-05-17 21:29:46,548 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520586257_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:46,548 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D5DA12156D9A499B9C11173561BC637D, taskAttemptId=attempt_1463520586257_0001_1_01_000000_0, startTime=1463520586509, finishTime=1463520586548, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520586540, Process_Duration=6, Process_End_Time=1463520586546, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:46,549 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520586257_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:46,549 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520586257_0001_1_01_000000_0
2016-05-17 21:29:46,549 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:46,549 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:46,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:TASK_FINISHED]: vertexName=D5DA12156D9A499B9C11173561BC637D, taskId=task_1463520586257_0001_1_01_000000, startTime=1463520586509, finishTime=1463520586549, timeTaken=40, status=SUCCEEDED, successfulAttemptID=attempt_1463520586257_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520586540, Process_Duration=6, Process_End_Time=1463520586546, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:46,549 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520586257_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:46,549 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D] : 1
2016-05-17 21:29:46,550 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:46,550 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D]
2016-05-17 21:29:46,550 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:VERTEX_FINISHED]: vertexName=D5DA12156D9A499B9C11173561BC637D, vertexId=vertex_1463520586257_0001_1_01, initRequestedTime=1463520586415, initedTime=1463520586422, startRequestedTime=1463520586428, startedTime=1463520586428, finishTime=1463520586550, timeTaken=122, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520586540, Process_Duration=6, Process_End_Time=1463520586546, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520586509, firstTasksToStart=[ task_1463520586257_0001_1_01_000000 ], lastTaskFinishTime=1463520586548, lastTasksToFinish=[ task_1463520586257_0001_1_01_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520586257_0001_1_01_000000 ], longestDurationTasks=[ task_1463520586257_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:46,551 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:46,551 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586257_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:46,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520586540, Process_Duration=6, Process_End_Time=1463520586546, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:46,551 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520586257_0001_00_000002, stoppedTime=1463520586551, exitStatus=0
2016-05-17 21:29:46,551 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586257_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:46,551 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:46,551 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:46,551 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520586257_0001_1
2016-05-17 21:29:46,551 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520586257_0001_1
2016-05-17 21:29:46,551 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520586257_0001_1_00 [C938E68BD4F043F78126DDCD2033CB18]
2016-05-17 21:29:46,552 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 41CD43C99F6D46A18826E7888235ECB1 for vertex: vertex_1463520586257_0001_1_01 [D5DA12156D9A499B9C11173561BC637D]
2016-05-17 21:29:46,551 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:46,552 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:46,552 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520586257_0001_00_000002
2016-05-17 21:29:46,552 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520586257_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:46,552 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520586257_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:46,553 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520586257_0001_00_000002, Exiting
2016-05-17 21:29:46,553 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520586257_0001_00_000002
2016-05-17 21:29:46,553 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520586257_0001_00_000002
2016-05-17 21:29:46,553 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520586257_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:46,553 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520586257_0001_00_000002 completed successfully
2016-05-17 21:29:46,552 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:46,559 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520586257_0001_1, queueSize=0
2016-05-17 21:29:46,560 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586257_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520586257_0001_1, startTime=1463520586414, finishTime=1463520586558, timeTaken=144, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=345, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041173025, Process_Duration=8, Process_End_Time=2927041173033, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:46,560 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:46,560 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520586257_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:46,560 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520586257_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:46,560 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586257_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:46 Completed Dag: dag_1463520586257_0001_1
2016-05-17 21:29:46,560 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520586257_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:46,560 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520586257_0001_1_post
2016-05-17 21:29:46,560 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:46,560 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:46,560 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520586257_0001_00_000002
2016-05-17 21:29:46,567 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_'field'_'field', applicationId=application_1463520586257_0001
2016-05-17 21:29:46,568 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:46,568 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:46,568 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:46,568 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:46,568 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:46,568 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:46,569 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:46,569 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:46,569 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:46,569 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:46,569 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:46,569 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:46,569 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:46,573 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/24819D7C1AF24D4185DEADC4C5B4C490/BEA2C00810AA4A248E31716D4DDF0D52/.tez/application_1463520586257_0001
2016-05-17 21:29:46,574 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_'field'/_temporary
2016-05-17 21:29:46,574 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.333
2016-05-17 21:29:46,636 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_'field'
2016-05-17 21:29:46,655 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:46,655 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:46,682 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 21:29:46,689 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 21:29:46,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:46,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:46,690 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:46,702 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:46,706 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_RESULTS
2016-05-17 21:29:46,762 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:46,763 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:46,763 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_RESULTS"]
2016-05-17 21:29:46,763 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:46,763 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:46,763 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:46,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...tform/ALL_'field'_RESULTS
2016-05-17 21:29:46,765 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:46,777 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:46,781 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:46,781 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:46,781 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:46,791 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D7BD56435EC144C3880371F193EC5FE3/FFC95E5B015B4B579933439BAE7465C3/.tez/application_1463520586781_0001 doesn't exist and is created
2016-05-17 21:29:46,805 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:46,805 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:46,806 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D7BD56435EC144C3880371F193EC5FE3/FFC95E5B015B4B579933439BAE7465C3/.tez/application_1463520586781_0001
2016-05-17 21:29:46,808 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520586781_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:46,816 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:46,817 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:46,818 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:46,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:46,818 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D7BD56435EC144C3880371F193EC5FE3/FFC95E5B015B4B579933439BAE7465C3/.tez/application_1463520586781_0001/localmode-log-dir
2016-05-17 21:29:46,818 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D7BD56435EC144C3880371F193EC5FE3/FFC95E5B015B4B579933439BAE7465C3/.tez/application_1463520586781_0001/localmode-log-dir/history.txt.appattempt_1463520586781_0001_000000, maxErrors=10
2016-05-17 21:29:46,818 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:46,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520586781_0001_000000, appSubmitTime=1463520586807, launchTime=1463520586807
2016-05-17 21:29:46,823 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:46,824 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:46,824 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35181
2016-05-17 21:29:46,825 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:46,826 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:46,827 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:46,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520586781_0001_000000, startTime=1463520586827
2016-05-17 21:29:46,827 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:46,827 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:46,906 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:46,906 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:46,906 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_RESULTS, applicationId=application_1463520586781_0001, dagName=[D7BD56435EC144C3880371F193EC5FE3/FFC95E5B015B4B579933439BAE7465C3] every/(1/1) ...tform/ALL_'field'_RESULTS
2016-05-17 21:29:46,920 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D7BD56435EC144C3880371F193EC5FE3/FFC95E5B015B4B579933439BAE7465C3] every/(1/1) ...tform/ALL_'field'_RESULTS
2016-05-17 21:29:46,921 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520586781_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D7BD56435EC144C3880371F193EC5FE3/FFC95E5B015B4B579933439BAE7465C3/.tez/application_1463520586781_0001/localmode-log-dir/dag_1463520586781_0001_1.dot
2016-05-17 21:29:46,921 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D7BD56435EC144C3880371F193EC5FE3/FFC95E5B015B4B579933439BAE7465C3/.tez/application_1463520586781_0001/localmode-log-dir/dag_1463520586781_0001_1-tez-dag.pb.txt
2016-05-17 21:29:46,926 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520586781_0001_1
2016-05-17 21:29:46,927 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:46,927 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D7BD56435EC144C3880371F193EC5FE3/FFC95E5B015B4B579933439BAE7465C3] every/(1/1) ...tform/ALL_'field'_RESULTS
2016-05-17 21:29:46 Running Dag: dag_1463520586781_0001_1
2016-05-17 21:29:46,936 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:46,936 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520586781_0001_1, submitTime=1463520586920
2016-05-17 21:29:46,937 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F78990CBF83A4F808A9D7767E70A075E
2016-05-17 21:29:46,937 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:46,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:46,938 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520586781_0001_1, initTime=1463520586936
2016-05-17 21:29:46,938 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520586781_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:46,939 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:DAG_STARTED]: dagID=dag_1463520586781_0001_1, startTime=1463520586939
2016-05-17 21:29:46,939 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_RESULTS, applicationId=application_1463520586781_0001, dagName=[D7BD56435EC144C3880371F193EC5FE3/FFC95E5B015B4B579933439BAE7465C3] every/(1/1) ...tform/ALL_'field'_RESULTS
2016-05-17 21:29:46,939 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:46,939 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520586781_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:46,939 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DA4A972B8CDD4A828021B536DE4F628E : {2667999AAF534DB4A2094405FD7A500C={InputName=2667999AAF534DB4A2094405FD7A500C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:46,939 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2667999AAF534DB4A2094405FD7A500C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:46,939 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E]
2016-05-17 21:29:46,940 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E]
2016-05-17 21:29:46,940 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E]
2016-05-17 21:29:46,940 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E]. Starting root input initializers: 1
2016-05-17 21:29:46,939 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:46,940 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:46,940 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E]
2016-05-17 21:29:46,941 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2667999AAF534DB4A2094405FD7A500C on vertex vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E]
2016-05-17 21:29:46,943 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:46,943 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:46,943 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E]
2016-05-17 21:29:46,943 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E]
2016-05-17 21:29:46,943 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E]
2016-05-17 21:29:46,944 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=083DD97634124823A6A74DBB8DFD12AD, vertexId=vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:46,944 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=083DD97634124823A6A74DBB8DFD12AD, vertexId=vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E]
2016-05-17 21:29:46,944 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2667999AAF534DB4A2094405FD7A500C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:46,945 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:46,945 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=083DD97634124823A6A74DBB8DFD12AD, vertexName=F78990CBF83A4F808A9D7767E70A075E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:46,945 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=083DD97634124823A6A74DBB8DFD12AD, vertexId=vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E]
2016-05-17 21:29:46,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F78990CBF83A4F808A9D7767E70A075E, vertexId=vertex_1463520586781_0001_1_01, initRequestedTime=1463520586940, initedTime=1463520586946, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:46,946 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:46,945 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:46,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:46,948 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:46,948 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2667999AAF534DB4A2094405FD7A500C on vertex vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E]
2016-05-17 21:29:46,948 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:46,948 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2667999AAF534DB4A2094405FD7A500C=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:46,948 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E] parallelism set to 1
2016-05-17 21:29:46,949 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:46,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520586781_0001_1_00, eventCount=1
2016-05-17 21:29:46,949 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:46,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DA4A972B8CDD4A828021B536DE4F628E, vertexId=vertex_1463520586781_0001_1_00, initRequestedTime=1463520586939, initedTime=1463520586949, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:46,949 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E] with distanceFromRoot: 0
2016-05-17 21:29:46,950 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:46,950 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DA4A972B8CDD4A828021B536DE4F628E
2016-05-17 21:29:46,950 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E]
2016-05-17 21:29:46,950 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:46,951 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520586781_0001_1_00, startRequestedTime=1463520586950, startedTime=1463520586950
2016-05-17 21:29:46,951 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:46,951 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:46,952 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:TASK_STARTED]: vertexName=DA4A972B8CDD4A828021B536DE4F628E, taskId=task_1463520586781_0001_1_00_000000, scheduledTime=1463520586951, launchTime=1463520586951
2016-05-17 21:29:46,953 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520586781_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:46,953 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520586781_0001_1_00 for vertex: vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E] numStartedSources: 1 numSources: 1
2016-05-17 21:29:46,953 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E] with distanceFromRoot: 1
2016-05-17 21:29:46,953 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E]
2016-05-17 21:29:46,953 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: DA4A972B8CDD4A828021B536DE4F628E in vertex: F78990CBF83A4F808A9D7767E70A075E
2016-05-17 21:29:46,954 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F78990CBF83A4F808A9D7767E70A075E with 1 source tasks and 1 pending tasks
2016-05-17 21:29:46,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:46,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520586781_0001_1_01, startRequestedTime=1463520586953, startedTime=1463520586953
2016-05-17 21:29:46,954 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:46,955 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D7BD56435EC144C3880371F193EC5FE3/FFC95E5B015B4B579933439BAE7465C3] every/(1/1) ...tform/ALL_'field'_RESULTS, VertexName: DA4A972B8CDD4A828021B536DE4F628E, VertexParallelism: 1, TaskAttemptID:attempt_1463520586781_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2667999AAF534DB4A2094405FD7A500C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F78990CBF83A4F808A9D7767E70A075E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:46,955 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520586781_0001, with dag id: dag_1463520586781_0001_1
2016-05-17 21:29:46,955 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520586781_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:46,955 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:46,956 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:46,956 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:46,956 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:46,957 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520586781_0001_00_000001
2016-05-17 21:29:46,957 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586781_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:46,959 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520586781_0001_00_000001
2016-05-17 21:29:46,959 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:46,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:46,960 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586781_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:46,960 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520586781_0001_00_000001, launchTime=1463520586960
2016-05-17 21:29:46,961 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520586781_0001_00_000001
2016-05-17 21:29:46,961 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520586781_0001_1_00_000000_0] to container: [container_1463520586781_0001_00_000001]
2016-05-17 21:29:46,962 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586781_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:46,962 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520586781_0001_00_000001 given task: attempt_1463520586781_0001_1_00_000000_0
2016-05-17 21:29:46,962 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520586781_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520586781_0001_1_00_000000_0
2016-05-17 21:29:46 Starting to run new task attempt: attempt_1463520586781_0001_1_00_000000_0
2016-05-17 21:29:46,962 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520586781_0001_1_00_000000_0
2016-05-17 21:29:46,962 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:46,962 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:46,962 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:46,963 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:46,963 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:46,963 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:46,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D7BD56435EC144C3880371F193EC5FE3/FFC95E5B015B4B579933439BAE7465C3] every/(1/1) ...tform/ALL_'field'_RESULTS, VertexName: DA4A972B8CDD4A828021B536DE4F628E, VertexParallelism: 1, TaskAttemptID:attempt_1463520586781_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2667999AAF534DB4A2094405FD7A500C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F78990CBF83A4F808A9D7767E70A075E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:46,963 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:46,963 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520586781_0001_1_00_000000_0
2016-05-17 21:29:46,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:46,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:46,964 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520586781_0001_1_00_000000_0] started. Is using containerId: [container_1463520586781_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:46,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2667999AAF534DB4A2094405FD7A500C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:46,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F78990CBF83A4F808A9D7767E70A075E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:46,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:46,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:46,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F78990CBF83A4F808A9D7767E70A075E
2016-05-17 21:29:46,964 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520586781_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:46,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DA4A972B8CDD4A828021B536DE4F628E, taskAttemptId=attempt_1463520586781_0001_1_00_000000_0, startTime=1463520586962, containerId=container_1463520586781_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520586781_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:46,964 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:46,966 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520586781_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:46,967 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:46,967 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F78990CBF83A4F808A9D7767E70A075E
2016-05-17 21:29:46,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F78990CBF83A4F808A9D7767E70A075E
2016-05-17 21:29:46,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:46,967 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:46,968 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520586781_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:46,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2667999AAF534DB4A2094405FD7A500C
2016-05-17 21:29:46,970 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2667999AAF534DB4A2094405FD7A500C
2016-05-17 21:29:46,970 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DA4A972B8CDD4A828021B536DE4F628E, ordinal: 0
2016-05-17 21:29:46,970 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA4A972B8CDD4A828021B536DE4F628E, mem on start (mb), free: 105, total: 364, max: 672
2016-05-17 21:29:46,970 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:46,971 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:46,971 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:46,972 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:46,972 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2667999AAF534DB4A2094405FD7A500C
2016-05-17 21:29:46,972 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2667999AAF534DB4A2094405FD7A500C
2016-05-17 21:29:46,972 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:46,973 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:46,974 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:46,974 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:46,974 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:46,974 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:46,974 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:46,974 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F78990CBF83A4F808A9D7767E70A075E, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:46,975 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2667999AAF534DB4A2094405FD7A500C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:46,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:46,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2667999AAF534DB4A2094405FD7A500C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:46,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2667999AAF534DB4A2094405FD7A500C
2016-05-17 21:29:46,975 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:46,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2667999AAF534DB4A2094405FD7A500C
2016-05-17 21:29:46,975 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:46,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:46,976 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:46,976 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520586781_0001_1_00_000000_0
2016-05-17 21:29:46,976 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:46,977 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@550b4521
2016-05-17 21:29:46,978 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@550b4521
2016-05-17 21:29:46,978 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:46,978 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:46,980 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:46,980 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2667999AAF534DB4A2094405FD7A500C
2016-05-17 21:29:46,980 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 57DD847F2EA4490B82606C1403018A05
2016-05-17 21:29:46,980 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 57DD847F2EA4490B82606C1403018A05
2016-05-17 21:29:46,981 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:46,983 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:46,983 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@6cdd7efd; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@41bc4be1; comparator=cascading.tuple.hadoop.util.TupleComparator@3119c27e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:46,983 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:46,984 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:46,984 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:47,006 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:47,006 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:47,007 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:47,007 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:47,007 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:47,008 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:47,008 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:47,009 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DA4A972B8CDD4A828021B536DE4F628E, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:47,010 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520586781_0001_1_00_000000_0
2016-05-17 21:29:47,011 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA4A972B8CDD4A828021B536DE4F628E, mem on close (mb), free: 190, total: 353, max: 672
2016-05-17 21:29:47,011 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:47,011 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:47,011 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:47,011 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:47,012 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:47,012 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:47,014 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:47 Completed running task attempt: attempt_1463520586781_0001_1_00_000000_0
2016-05-17 21:29:47,016 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520586781_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:47,017 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520586781_0001_1_00, eventCount=1
2016-05-17 21:29:47,017 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370147328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520587009, Process_Duration=1, Process_End_Time=1463520587010, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:47,017 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DA4A972B8CDD4A828021B536DE4F628E, taskAttemptId=attempt_1463520586781_0001_1_00_000000_0, startTime=1463520586962, finishTime=1463520587017, timeTaken=55, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370147328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520587009, Process_Duration=1, Process_End_Time=1463520587010, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:47,017 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:47,018 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:47,018 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:47,018 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520586781_0001_00_000001
2016-05-17 21:29:47,018 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520586781_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:47,018 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:47,019 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520586781_0001_1_00_000000_0
2016-05-17 21:29:47,019 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:47,019 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:TASK_FINISHED]: vertexName=DA4A972B8CDD4A828021B536DE4F628E, taskId=task_1463520586781_0001_1_00_000000, startTime=1463520586962, finishTime=1463520587019, timeTaken=57, status=SUCCEEDED, successfulAttemptID=attempt_1463520586781_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370147328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520587009, Process_Duration=1, Process_End_Time=1463520587010, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:47,019 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520586781_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:47,020 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E] : 1
2016-05-17 21:29:47,020 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E]
2016-05-17 21:29:47,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:47,021 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:VERTEX_FINISHED]: vertexName=DA4A972B8CDD4A828021B536DE4F628E, vertexId=vertex_1463520586781_0001_1_00, initRequestedTime=1463520586939, initedTime=1463520586949, startRequestedTime=1463520586950, startedTime=1463520586950, finishTime=1463520587020, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370147328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520587009, Process_Duration=1, Process_End_Time=1463520587010, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520586962, firstTasksToStart=[ task_1463520586781_0001_1_00_000000 ], lastTaskFinishTime=1463520587017, lastTasksToFinish=[ task_1463520586781_0001_1_00_000000 ], minTaskDuration=55, maxTaskDuration=55, avgTaskDuration=55.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520586781_0001_1_00_000000 ], longestDurationTasks=[ task_1463520586781_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:47,021 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:47,022 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586781_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:47,022 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520586781_0001_00_000001, stoppedTime=1463520587022, exitStatus=0
2016-05-17 21:29:47,022 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:47,022 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586781_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:47,023 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E] attempt: attempt_1463520586781_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:47,023 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F78990CBF83A4F808A9D7767E70A075E
2016-05-17 21:29:47,023 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E]
2016-05-17 21:29:47,023 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:47,023 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520586781_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:47,024 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:TASK_STARTED]: vertexName=F78990CBF83A4F808A9D7767E70A075E, taskId=task_1463520586781_0001_1_01_000000, scheduledTime=1463520587023, launchTime=1463520587023
2016-05-17 21:29:47,024 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:47,024 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520586781_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:47,024 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586781_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:47,024 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E]
2016-05-17 21:29:47,024 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D7BD56435EC144C3880371F193EC5FE3/FFC95E5B015B4B579933439BAE7465C3] every/(1/1) ...tform/ALL_'field'_RESULTS, VertexName: F78990CBF83A4F808A9D7767E70A075E, VertexParallelism: 1, TaskAttemptID:attempt_1463520586781_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DA4A972B8CDD4A828021B536DE4F628E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=083DD97634124823A6A74DBB8DFD12AD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:47,025 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520586781_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:47,025 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:47,025 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520586781_0001_00_000002
2016-05-17 21:29:47,025 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586781_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:47,027 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520586781_0001_00_000002
2016-05-17 21:29:47,027 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:47,027 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586781_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:47,027 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:47,027 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520586781_0001_00_000002, launchTime=1463520587027
2016-05-17 21:29:47,028 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520586781_0001_00_000002
2016-05-17 21:29:47,028 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520586781_0001_1_01_000000_0] to container: [container_1463520586781_0001_00_000002]
2016-05-17 21:29:47,028 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586781_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:47,029 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520586781_0001_1_01_000000_0] started. Is using containerId: [container_1463520586781_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:47,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F78990CBF83A4F808A9D7767E70A075E, taskAttemptId=attempt_1463520586781_0001_1_01_000000_0, startTime=1463520587029, containerId=container_1463520586781_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520586781_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:47,029 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:47,029 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520586781_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:47,029 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520586781_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:47,029 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520586781_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:47,029 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520586781_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:47,030 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520586781_0001_00_000001, Exiting
2016-05-17 21:29:47,030 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520586781_0001_00_000001
2016-05-17 21:29:47,030 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520586781_0001_00_000001
2016-05-17 21:29:47,030 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520586781_0001_00_000002 given task: attempt_1463520586781_0001_1_01_000000_0
2016-05-17 21:29:47,030 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520586781_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520586781_0001_1_01_000000_0
2016-05-17 21:29:47 Starting to run new task attempt: attempt_1463520586781_0001_1_01_000000_0
2016-05-17 21:29:47,031 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520586781_0001_1_01_000000_0
2016-05-17 21:29:47,031 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:47,031 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520586781_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:47,031 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520586781_0001_00_000001 completed successfully
2016-05-17 21:29:47,031 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:47,031 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:47,031 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586781_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:47,032 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:47,032 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:47,032 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:47,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D7BD56435EC144C3880371F193EC5FE3/FFC95E5B015B4B579933439BAE7465C3] every/(1/1) ...tform/ALL_'field'_RESULTS, VertexName: F78990CBF83A4F808A9D7767E70A075E, VertexParallelism: 1, TaskAttemptID:attempt_1463520586781_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DA4A972B8CDD4A828021B536DE4F628E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=083DD97634124823A6A74DBB8DFD12AD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:47,033 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:47,034 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520586781_0001_1_01_000000_0
2016-05-17 21:29:47,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:47,034 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520586781_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:47,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:47,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:47,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=083DD97634124823A6A74DBB8DFD12AD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:47,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=DA4A972B8CDD4A828021B536DE4F628E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:47,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:47,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 083DD97634124823A6A74DBB8DFD12AD
2016-05-17 21:29:47,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:47,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: DA4A972B8CDD4A828021B536DE4F628E
2016-05-17 21:29:47,036 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:47,037 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 083DD97634124823A6A74DBB8DFD12AD
2016-05-17 21:29:47,038 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:47,038 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:47,038 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: DA4A972B8CDD4A828021B536DE4F628E
2016-05-17 21:29:47,038 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: DA4A972B8CDD4A828021B536DE4F628E
2016-05-17 21:29:47,039 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:47,042 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F78990CBF83A4F808A9D7767E70A075E, ordinal: 1
2016-05-17 21:29:47,042 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F78990CBF83A4F808A9D7767E70A075E, mem on start (mb), free: 184, total: 353, max: 672
2016-05-17 21:29:47,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:47,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:47,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:47,046 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:47,046 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 083DD97634124823A6A74DBB8DFD12AD
2016-05-17 21:29:47,046 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:47,046 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:47,046 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:47,047 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:47,047 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:47,047 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:47,047 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:47,047 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 083DD97634124823A6A74DBB8DFD12AD, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:47,047 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, DA4A972B8CDD4A828021B536DE4F628E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:47,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:47,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: DA4A972B8CDD4A828021B536DE4F628E being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:47,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:47,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:47,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: DA4A972B8CDD4A828021B536DE4F628E
2016-05-17 21:29:47,048 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:47,048 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:47,048 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: DA4A972B8CDD4A828021B536DE4F628E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:47,048 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:47,049 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:47,049 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:47,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: DA4A972B8CDD4A828021B536DE4F628E
2016-05-17 21:29:47,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:47,050 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520586781_0001_1_01_000000_0
2016-05-17 21:29:47,050 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520586781_0001_1_00_000000_0_10243, runDuration: 50258, ]
2016-05-17 21:29:47,052 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:47,052 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 57DD847F2EA4490B82606C1403018A05
2016-05-17 21:29:47,052 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_RESULTS"], id: 083DD97634124823A6A74DBB8DFD12AD
2016-05-17 21:29:47,052 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_RESULTS"]
2016-05-17 21:29:47,052 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520586781_0001_1_00_000000_0_10243] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520587052, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:47,053 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : DA4A972B8CDD4A828021B536DE4F628E
2016-05-17 21:29:47,053 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 57DD847F2EA4490B82606C1403018A05, for 1 inputs
2016-05-17 21:29:47,054 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:47,054 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [DA4A972B8CDD4A828021B536DE4F628E] #41 in 3ms
2016-05-17 21:29:47,054 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [DA4A972B8CDD4A828021B536DE4F628E] #41, status:false, isInterrupted:false
2016-05-17 21:29:47,054 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:47,055 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:47,055 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:47,055 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:47,055 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:47,056 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:47,056 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : DA4A972B8CDD4A828021B536DE4F628E
2016-05-17 21:29:47,056 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:47,056 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F78990CBF83A4F808A9D7767E70A075E, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:47,057 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:47,059 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@6d7bedfa; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@6344f03d
2016-05-17 21:29:47,061 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:47,061 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:47,061 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520586781_0001_1_01_000000_0
2016-05-17 21:29:47,061 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520586781_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:47,062 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520586781_0001_1_01_000000_0
2016-05-17 21:29:47,062 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F78990CBF83A4F808A9D7767E70A075E, mem on close (mb), free: 181, total: 353, max: 672
2016-05-17 21:29:47,062 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: DA4A972B8CDD4A828021B536DE4F628E
2016-05-17 21:29:47,062 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:47,063 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:47,063 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:47,063 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:47,063 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:47,063 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:47 Completed running task attempt: attempt_1463520586781_0001_1_01_000000_0
2016-05-17 21:29:47,064 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520586781_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:47,064 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:47,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F78990CBF83A4F808A9D7767E70A075E, taskAttemptId=attempt_1463520586781_0001_1_01_000000_0, startTime=1463520587029, finishTime=1463520587064, timeTaken=35, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370147328, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520587056, Process_Duration=6, Process_End_Time=1463520587062, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:47,065 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520586781_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:47,065 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520586781_0001_1_01_000000_0
2016-05-17 21:29:47,065 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:47,065 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:47,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:TASK_FINISHED]: vertexName=F78990CBF83A4F808A9D7767E70A075E, taskId=task_1463520586781_0001_1_01_000000, startTime=1463520587029, finishTime=1463520587065, timeTaken=36, status=SUCCEEDED, successfulAttemptID=attempt_1463520586781_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370147328, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520587056, Process_Duration=6, Process_End_Time=1463520587062, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:47,065 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520586781_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:47,066 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E] : 1
2016-05-17 21:29:47,066 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E]
2016-05-17 21:29:47,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:47,066 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:VERTEX_FINISHED]: vertexName=F78990CBF83A4F808A9D7767E70A075E, vertexId=vertex_1463520586781_0001_1_01, initRequestedTime=1463520586940, initedTime=1463520586946, startRequestedTime=1463520586953, startedTime=1463520586953, finishTime=1463520587066, timeTaken=113, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370147328, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520587056, Process_Duration=6, Process_End_Time=1463520587062, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520587029, firstTasksToStart=[ task_1463520586781_0001_1_01_000000 ], lastTaskFinishTime=1463520587064, lastTasksToFinish=[ task_1463520586781_0001_1_01_000000 ], minTaskDuration=35, maxTaskDuration=35, avgTaskDuration=35.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520586781_0001_1_01_000000 ], longestDurationTasks=[ task_1463520586781_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:47,066 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:47,066 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586781_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:47,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=370147328, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520587056, Process_Duration=6, Process_End_Time=1463520587062, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:47,067 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520586781_0001_00_000002, stoppedTime=1463520587067, exitStatus=0
2016-05-17 21:29:47,067 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:47,067 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:47,067 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:47,067 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586781_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:47,067 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520586781_0001_00_000002
2016-05-17 21:29:47,067 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:47,067 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520586781_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:47,067 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520586781_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:47,067 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520586781_0001_1
2016-05-17 21:29:47,067 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520586781_0001_00_000002, Exiting
2016-05-17 21:29:47,067 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520586781_0001_1
2016-05-17 21:29:47,068 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520586781_0001_1_00 [DA4A972B8CDD4A828021B536DE4F628E]
2016-05-17 21:29:47,067 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520586781_0001_00_000002
2016-05-17 21:29:47,068 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520586781_0001_00_000002
2016-05-17 21:29:47,068 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 083DD97634124823A6A74DBB8DFD12AD for vertex: vertex_1463520586781_0001_1_01 [F78990CBF83A4F808A9D7767E70A075E]
2016-05-17 21:29:47,068 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520586781_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:47,068 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520586781_0001_00_000002 completed successfully
2016-05-17 21:29:47,075 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520586781_0001_1, queueSize=0
2016-05-17 21:29:47,075 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:47,075 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520586781_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520586781_0001_1, startTime=1463520586939, finishTime=1463520587074, timeTaken=135, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=1381, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=740294656, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041174065, Process_Duration=7, Process_End_Time=2927041174072, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:47,075 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520586781_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:47,075 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520586781_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:47,075 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520586781_0001_00_000002
2016-05-17 21:29:47,076 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520586781_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:47 Completed Dag: dag_1463520586781_0001_1
2016-05-17 21:29:47,076 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520586781_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:47,076 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520586781_0001_1_post
2016-05-17 21:29:47,076 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:47,076 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:47,082 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_'field'_RESULTS, applicationId=application_1463520586781_0001
2016-05-17 21:29:47,084 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:47,084 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:47,084 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:47,084 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:47,084 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:47,085 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:47,085 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:47,085 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:47,085 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:47,086 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:47,086 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:47,086 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:47,086 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:47,092 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/D7BD56435EC144C3880371F193EC5FE3/FFC95E5B015B4B579933439BAE7465C3/.tez/application_1463520586781_0001
2016-05-17 21:29:47,092 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_RESULTS/_temporary
2016-05-17 21:29:47,094 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.330
2016-05-17 21:29:47,153 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_RESULTS
2016-05-17 21:29:47,173 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:47,174 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:47,207 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 21:29:47,211 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 21:29:47,212 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:47,212 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:47,212 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:47,226 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:47,232 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_ALL
2016-05-17 21:29:47,272 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:47,273 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:47,273 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_ALL"]
2016-05-17 21:29:47,273 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:47,273 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:47,273 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:47,274 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...yplatform/ALL_'field'_ALL
2016-05-17 21:29:47,274 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:47,284 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:47,289 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:47,289 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:47,289 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:47,298 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7B74C835F9894FB68EEF755EF7D1B574/F5A79B0EB95F404B97D14956E904D931/.tez/application_1463520587289_0001 doesn't exist and is created
2016-05-17 21:29:47,314 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:47,314 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:47,314 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7B74C835F9894FB68EEF755EF7D1B574/F5A79B0EB95F404B97D14956E904D931/.tez/application_1463520587289_0001
2016-05-17 21:29:47,316 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520587289_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:47,325 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:47,326 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:47,327 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:47,327 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:47,327 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7B74C835F9894FB68EEF755EF7D1B574/F5A79B0EB95F404B97D14956E904D931/.tez/application_1463520587289_0001/localmode-log-dir
2016-05-17 21:29:47,327 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7B74C835F9894FB68EEF755EF7D1B574/F5A79B0EB95F404B97D14956E904D931/.tez/application_1463520587289_0001/localmode-log-dir/history.txt.appattempt_1463520587289_0001_000000, maxErrors=10
2016-05-17 21:29:47,327 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:47,327 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520587289_0001_000000, appSubmitTime=1463520587316, launchTime=1463520587316
2016-05-17 21:29:47,329 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:47,330 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:47,330 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:47,331 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:47,332 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:50495
2016-05-17 21:29:47,334 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:47,335 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520587289_0001_000000, startTime=1463520587335
2016-05-17 21:29:47,335 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:47,335 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:47,414 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:47,414 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:47,415 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/ALL_'field'_ALL, applicationId=application_1463520587289_0001, dagName=[7B74C835F9894FB68EEF755EF7D1B574/F5A79B0EB95F404B97D14956E904D931] every/(1/1) ...yplatform/ALL_'field'_ALL
2016-05-17 21:29:47,429 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7B74C835F9894FB68EEF755EF7D1B574/F5A79B0EB95F404B97D14956E904D931] every/(1/1) ...yplatform/ALL_'field'_ALL
2016-05-17 21:29:47,430 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520587289_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7B74C835F9894FB68EEF755EF7D1B574/F5A79B0EB95F404B97D14956E904D931/.tez/application_1463520587289_0001/localmode-log-dir/dag_1463520587289_0001_1.dot
2016-05-17 21:29:47,430 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7B74C835F9894FB68EEF755EF7D1B574/F5A79B0EB95F404B97D14956E904D931/.tez/application_1463520587289_0001/localmode-log-dir/dag_1463520587289_0001_1-tez-dag.pb.txt
2016-05-17 21:29:47,436 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520587289_0001_1
2016-05-17 21:29:47,436 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:47,436 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7B74C835F9894FB68EEF755EF7D1B574/F5A79B0EB95F404B97D14956E904D931] every/(1/1) ...yplatform/ALL_'field'_ALL
2016-05-17 21:29:47 Running Dag: dag_1463520587289_0001_1
2016-05-17 21:29:47,447 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520587289_0001_1, submitTime=1463520587429
2016-05-17 21:29:47,447 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:47,447 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CCA4319F6A254CA0B22ED56520ABAE14
2016-05-17 21:29:47,448 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:47,448 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520587289_0001_1, initTime=1463520587447
2016-05-17 21:29:47,448 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:47,448 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520587289_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:47,448 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:DAG_STARTED]: dagID=dag_1463520587289_0001_1, startTime=1463520587448
2016-05-17 21:29:47,449 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:47,449 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520587289_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:47,449 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BFFC3FC0499742D0AE1F11350B15CE23 : {A32D47984E324308856D23BB2E802260={InputName=A32D47984E324308856D23BB2E802260}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:47,449 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A32D47984E324308856D23BB2E802260, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:47,448 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:47,449 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23]
2016-05-17 21:29:47,449 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23]
2016-05-17 21:29:47,449 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23]
2016-05-17 21:29:47,449 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23]. Starting root input initializers: 1
2016-05-17 21:29:47,450 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:47,450 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14]
2016-05-17 21:29:47,451 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/ALL_'field'_ALL, applicationId=application_1463520587289_0001, dagName=[7B74C835F9894FB68EEF755EF7D1B574/F5A79B0EB95F404B97D14956E904D931] every/(1/1) ...yplatform/ALL_'field'_ALL
2016-05-17 21:29:47,452 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A32D47984E324308856D23BB2E802260 on vertex vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23]
2016-05-17 21:29:47,453 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:47,453 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14]
2016-05-17 21:29:47,453 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14]
2016-05-17 21:29:47,453 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14]
2016-05-17 21:29:47,453 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6EBF5AE60E3E4F7D9031C0A3C18A7954, vertexId=vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:47,454 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6EBF5AE60E3E4F7D9031C0A3C18A7954, vertexId=vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14]
2016-05-17 21:29:47,455 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:47,455 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A32D47984E324308856D23BB2E802260 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:47,455 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6EBF5AE60E3E4F7D9031C0A3C18A7954, vertexName=CCA4319F6A254CA0B22ED56520ABAE14, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:47,456 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6EBF5AE60E3E4F7D9031C0A3C18A7954, vertexId=vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14]
2016-05-17 21:29:47,456 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:47,456 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:47,457 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:47,457 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CCA4319F6A254CA0B22ED56520ABAE14, vertexId=vertex_1463520587289_0001_1_01, initRequestedTime=1463520587450, initedTime=1463520587457, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:47,457 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:47,458 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:47,459 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A32D47984E324308856D23BB2E802260 on vertex vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23]
2016-05-17 21:29:47,459 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:47,459 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A32D47984E324308856D23BB2E802260=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:47,459 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23] parallelism set to 1
2016-05-17 21:29:47,460 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:47,460 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520587289_0001_1_00, eventCount=1
2016-05-17 21:29:47,461 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:47,461 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BFFC3FC0499742D0AE1F11350B15CE23, vertexId=vertex_1463520587289_0001_1_00, initRequestedTime=1463520587449, initedTime=1463520587460, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:47,461 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23] with distanceFromRoot: 0
2016-05-17 21:29:47,461 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:47,461 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BFFC3FC0499742D0AE1F11350B15CE23
2016-05-17 21:29:47,461 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23]
2016-05-17 21:29:47,462 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:47,462 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520587289_0001_1_00, startRequestedTime=1463520587461, startedTime=1463520587461
2016-05-17 21:29:47,462 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:47,463 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:47,463 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:TASK_STARTED]: vertexName=BFFC3FC0499742D0AE1F11350B15CE23, taskId=task_1463520587289_0001_1_00_000000, scheduledTime=1463520587463, launchTime=1463520587463
2016-05-17 21:29:47,463 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520587289_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:47,463 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520587289_0001_1_00 for vertex: vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14] numStartedSources: 1 numSources: 1
2016-05-17 21:29:47,464 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14] with distanceFromRoot: 1
2016-05-17 21:29:47,464 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23]
2016-05-17 21:29:47,464 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BFFC3FC0499742D0AE1F11350B15CE23 in vertex: CCA4319F6A254CA0B22ED56520ABAE14
2016-05-17 21:29:47,464 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CCA4319F6A254CA0B22ED56520ABAE14 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:47,465 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:47,465 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520587289_0001_1_01, startRequestedTime=1463520587464, startedTime=1463520587464
2016-05-17 21:29:47,465 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:47,466 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7B74C835F9894FB68EEF755EF7D1B574/F5A79B0EB95F404B97D14956E904D931] every/(1/1) ...yplatform/ALL_'field'_ALL, VertexName: BFFC3FC0499742D0AE1F11350B15CE23, VertexParallelism: 1, TaskAttemptID:attempt_1463520587289_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A32D47984E324308856D23BB2E802260, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CCA4319F6A254CA0B22ED56520ABAE14, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:47,466 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520587289_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:47,466 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:47,466 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520587289_0001, with dag id: dag_1463520587289_0001_1
2016-05-17 21:29:47,468 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:47,468 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:47,468 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:47,469 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520587289_0001_00_000001
2016-05-17 21:29:47,469 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587289_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:47,471 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520587289_0001_00_000001
2016-05-17 21:29:47,471 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:47,471 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587289_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:47,471 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:47,471 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520587289_0001_00_000001, launchTime=1463520587471
2016-05-17 21:29:47,473 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520587289_0001_00_000001
2016-05-17 21:29:47,474 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520587289_0001_1_00_000000_0] to container: [container_1463520587289_0001_00_000001]
2016-05-17 21:29:47,474 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587289_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:47,474 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520587289_0001_00_000001 given task: attempt_1463520587289_0001_1_00_000000_0
2016-05-17 21:29:47,474 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520587289_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520587289_0001_1_00_000000_0
2016-05-17 21:29:47,474 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520587289_0001_1_00_000000_0] started. Is using containerId: [container_1463520587289_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:47 Starting to run new task attempt: attempt_1463520587289_0001_1_00_000000_0
2016-05-17 21:29:47,474 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BFFC3FC0499742D0AE1F11350B15CE23, taskAttemptId=attempt_1463520587289_0001_1_00_000000_0, startTime=1463520587474, containerId=container_1463520587289_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520587289_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:47,474 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520587289_0001_1_00_000000_0
2016-05-17 21:29:47,475 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:47,475 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:47,475 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:47,475 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:47,475 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:47,475 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:47,475 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:47,475 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7B74C835F9894FB68EEF755EF7D1B574/F5A79B0EB95F404B97D14956E904D931] every/(1/1) ...yplatform/ALL_'field'_ALL, VertexName: BFFC3FC0499742D0AE1F11350B15CE23, VertexParallelism: 1, TaskAttemptID:attempt_1463520587289_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A32D47984E324308856D23BB2E802260, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CCA4319F6A254CA0B22ED56520ABAE14, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:47,475 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520587289_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:47,475 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:47,475 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520587289_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:47,476 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520587289_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:47,476 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520587289_0001_1_00_000000_0
2016-05-17 21:29:47,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:47,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:47,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A32D47984E324308856D23BB2E802260, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:47,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CCA4319F6A254CA0B22ED56520ABAE14, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:47,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:47,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:47,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:47,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CCA4319F6A254CA0B22ED56520ABAE14
2016-05-17 21:29:47,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A32D47984E324308856D23BB2E802260
2016-05-17 21:29:47,479 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A32D47984E324308856D23BB2E802260
2016-05-17 21:29:47,479 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:47,480 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CCA4319F6A254CA0B22ED56520ABAE14
2016-05-17 21:29:47,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CCA4319F6A254CA0B22ED56520ABAE14
2016-05-17 21:29:47,480 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:47,481 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:47,481 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A32D47984E324308856D23BB2E802260
2016-05-17 21:29:47,482 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A32D47984E324308856D23BB2E802260
2016-05-17 21:29:47,484 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:47,484 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:47,484 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BFFC3FC0499742D0AE1F11350B15CE23, ordinal: 0
2016-05-17 21:29:47,484 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BFFC3FC0499742D0AE1F11350B15CE23, mem on start (mb), free: 91, total: 353, max: 672
2016-05-17 21:29:47,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:47,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:47,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:47,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:47,485 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:47,485 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:47,485 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:47,486 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:47,486 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:47,486 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:47,486 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A32D47984E324308856D23BB2E802260, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:47,487 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CCA4319F6A254CA0B22ED56520ABAE14, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:47,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:47,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A32D47984E324308856D23BB2E802260 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:47,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A32D47984E324308856D23BB2E802260
2016-05-17 21:29:47,487 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:47,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A32D47984E324308856D23BB2E802260
2016-05-17 21:29:47,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:47,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:47,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:47,488 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520587289_0001_1_00_000000_0
2016-05-17 21:29:47,488 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:47,488 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@43f7cbeb
2016-05-17 21:29:47,489 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@43f7cbeb
2016-05-17 21:29:47,489 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:47,489 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:47,490 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:47,491 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: A32D47984E324308856D23BB2E802260
2016-05-17 21:29:47,491 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: EA47917DCBF3418E83CF3DCC5BD05A0B
2016-05-17 21:29:47,491 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] EA47917DCBF3418E83CF3DCC5BD05A0B
2016-05-17 21:29:47,492 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:47,494 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:47,494 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@2f62b1cd; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@78872208; comparator=cascading.tuple.hadoop.util.TupleComparator@6fcb7909; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:47,494 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:47,494 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:47,494 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:47,517 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:47,517 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:47,517 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:47,517 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:47,518 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:47,518 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:47,518 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:47,519 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BFFC3FC0499742D0AE1F11350B15CE23, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:29:47,520 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520587289_0001_1_00_000000_0
2016-05-17 21:29:47,521 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BFFC3FC0499742D0AE1F11350B15CE23, mem on close (mb), free: 198, total: 363, max: 672
2016-05-17 21:29:47,521 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:47,521 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:47,521 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:47,521 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:47,524 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:47 Completed running task attempt: attempt_1463520587289_0001_1_00_000000_0
2016-05-17 21:29:47,526 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520587289_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:47,527 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520587289_0001_1_00, eventCount=1
2016-05-17 21:29:47,527 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BFFC3FC0499742D0AE1F11350B15CE23, taskAttemptId=attempt_1463520587289_0001_1_00_000000_0, startTime=1463520587474, finishTime=1463520587527, timeTaken=53, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520587519, Process_Duration=1, Process_End_Time=1463520587520, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:47,527 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520587289_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:47,528 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:47,528 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520587289_0001_1_00_000000_0
2016-05-17 21:29:47,528 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:47,528 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:47,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:TASK_FINISHED]: vertexName=BFFC3FC0499742D0AE1F11350B15CE23, taskId=task_1463520587289_0001_1_00_000000, startTime=1463520587474, finishTime=1463520587528, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1463520587289_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520587519, Process_Duration=1, Process_End_Time=1463520587520, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:47,528 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520587289_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:47,528 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23] : 1
2016-05-17 21:29:47,528 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23]
2016-05-17 21:29:47,529 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:47,529 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:VERTEX_FINISHED]: vertexName=BFFC3FC0499742D0AE1F11350B15CE23, vertexId=vertex_1463520587289_0001_1_00, initRequestedTime=1463520587449, initedTime=1463520587460, startRequestedTime=1463520587461, startedTime=1463520587461, finishTime=1463520587528, timeTaken=67, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520587519, Process_Duration=1, Process_End_Time=1463520587520, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520587474, firstTasksToStart=[ task_1463520587289_0001_1_00_000000 ], lastTaskFinishTime=1463520587527, lastTasksToFinish=[ task_1463520587289_0001_1_00_000000 ], minTaskDuration=53, maxTaskDuration=53, avgTaskDuration=53.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520587289_0001_1_00_000000 ], longestDurationTasks=[ task_1463520587289_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:47,529 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:47,530 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520587519, Process_Duration=1, Process_End_Time=1463520587520, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:47,531 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:47,530 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587289_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:47,531 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:47,531 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:47,531 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520587289_0001_00_000001, stoppedTime=1463520587531, exitStatus=0
2016-05-17 21:29:47,531 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587289_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:47,531 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520587289_0001_00_000001
2016-05-17 21:29:47,531 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14] attempt: attempt_1463520587289_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:47,532 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CCA4319F6A254CA0B22ED56520ABAE14
2016-05-17 21:29:47,532 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14]
2016-05-17 21:29:47,532 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:47,532 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520587289_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:47,533 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:47,533 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:TASK_STARTED]: vertexName=CCA4319F6A254CA0B22ED56520ABAE14, taskId=task_1463520587289_0001_1_01_000000, scheduledTime=1463520587532, launchTime=1463520587532
2016-05-17 21:29:47,532 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520587289_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:47,533 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520587289_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:47,533 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520587289_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:47,533 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587289_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:47,533 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520587289_0001_00_000001, Exiting
2016-05-17 21:29:47,533 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520587289_0001_00_000001
2016-05-17 21:29:47,533 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520587289_0001_00_000001
2016-05-17 21:29:47,533 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14]
2016-05-17 21:29:47,534 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7B74C835F9894FB68EEF755EF7D1B574/F5A79B0EB95F404B97D14956E904D931] every/(1/1) ...yplatform/ALL_'field'_ALL, VertexName: CCA4319F6A254CA0B22ED56520ABAE14, VertexParallelism: 1, TaskAttemptID:attempt_1463520587289_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BFFC3FC0499742D0AE1F11350B15CE23, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6EBF5AE60E3E4F7D9031C0A3C18A7954, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:47,534 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520587289_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:47,534 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:47,534 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520587289_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:47,535 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520587289_0001_00_000002
2016-05-17 21:29:47,535 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587289_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:47,535 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520587289_0001_00_000001 completed successfully
2016-05-17 21:29:47,535 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587289_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:47,536 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520587289_0001_00_000002
2016-05-17 21:29:47,536 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:47,538 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:47,538 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587289_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:47,539 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520587289_0001_00_000002, launchTime=1463520587538
2016-05-17 21:29:47,539 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520587289_0001_00_000002
2016-05-17 21:29:47,540 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520587289_0001_1_01_000000_0] to container: [container_1463520587289_0001_00_000002]
2016-05-17 21:29:47,540 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587289_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:47,540 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520587289_0001_1_01_000000_0] started. Is using containerId: [container_1463520587289_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:47,540 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CCA4319F6A254CA0B22ED56520ABAE14, taskAttemptId=attempt_1463520587289_0001_1_01_000000_0, startTime=1463520587540, containerId=container_1463520587289_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520587289_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:47,540 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520587289_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:47,541 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520587289_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:47,540 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:47,541 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520587289_0001_00_000002 given task: attempt_1463520587289_0001_1_01_000000_0
2016-05-17 21:29:47,541 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520587289_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520587289_0001_1_01_000000_0
2016-05-17 21:29:47 Starting to run new task attempt: attempt_1463520587289_0001_1_01_000000_0
2016-05-17 21:29:47,541 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520587289_0001_1_01_000000_0
2016-05-17 21:29:47,541 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:47,541 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:47,542 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:47,542 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:47,542 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:47,542 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:47,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7B74C835F9894FB68EEF755EF7D1B574/F5A79B0EB95F404B97D14956E904D931] every/(1/1) ...yplatform/ALL_'field'_ALL, VertexName: CCA4319F6A254CA0B22ED56520ABAE14, VertexParallelism: 1, TaskAttemptID:attempt_1463520587289_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BFFC3FC0499742D0AE1F11350B15CE23, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6EBF5AE60E3E4F7D9031C0A3C18A7954, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:47,543 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:47,543 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520587289_0001_1_01_000000_0
2016-05-17 21:29:47,543 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:47,543 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:47,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:47,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BFFC3FC0499742D0AE1F11350B15CE23, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:47,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:47,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BFFC3FC0499742D0AE1F11350B15CE23
2016-05-17 21:29:47,545 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:47,545 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:47,546 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BFFC3FC0499742D0AE1F11350B15CE23
2016-05-17 21:29:47,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BFFC3FC0499742D0AE1F11350B15CE23
2016-05-17 21:29:47,546 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:47,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6EBF5AE60E3E4F7D9031C0A3C18A7954, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:47,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:47,548 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6EBF5AE60E3E4F7D9031C0A3C18A7954
2016-05-17 21:29:47,549 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:47,549 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6EBF5AE60E3E4F7D9031C0A3C18A7954
2016-05-17 21:29:47,551 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CCA4319F6A254CA0B22ED56520ABAE14, ordinal: 1
2016-05-17 21:29:47,551 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CCA4319F6A254CA0B22ED56520ABAE14, mem on start (mb), free: 193, total: 363, max: 672
2016-05-17 21:29:47,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:47,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:47,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:47,551 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520587289_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:47,560 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:47,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6EBF5AE60E3E4F7D9031C0A3C18A7954
2016-05-17 21:29:47,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:47,561 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:47,561 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:47,561 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:47,561 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:47,561 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:47,562 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:47,562 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BFFC3FC0499742D0AE1F11350B15CE23, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:47,562 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6EBF5AE60E3E4F7D9031C0A3C18A7954, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:47,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:47,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BFFC3FC0499742D0AE1F11350B15CE23 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:47,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BFFC3FC0499742D0AE1F11350B15CE23
2016-05-17 21:29:47,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:47,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:47,562 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:47,563 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:47,563 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BFFC3FC0499742D0AE1F11350B15CE23 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:47,563 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:47,563 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:47,563 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:47,564 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BFFC3FC0499742D0AE1F11350B15CE23
2016-05-17 21:29:47,564 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:47,565 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520587289_0001_1_01_000000_0
2016-05-17 21:29:47,567 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:47,568 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520587289_0001_1_00_000000_0_10249, runDuration: 47123, ]
2016-05-17 21:29:47,568 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: EA47917DCBF3418E83CF3DCC5BD05A0B
2016-05-17 21:29:47,568 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_ALL"], id: 6EBF5AE60E3E4F7D9031C0A3C18A7954
2016-05-17 21:29:47,568 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_ALL"]
2016-05-17 21:29:47,569 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] EA47917DCBF3418E83CF3DCC5BD05A0B, for 1 inputs
2016-05-17 21:29:47,569 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520587289_0001_1_00_000000_0_10249] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520587569, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:47,569 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BFFC3FC0499742D0AE1F11350B15CE23
2016-05-17 21:29:47,569 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:47,569 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [BFFC3FC0499742D0AE1F11350B15CE23] #42 in 1ms
2016-05-17 21:29:47,569 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BFFC3FC0499742D0AE1F11350B15CE23] #42, status:false, isInterrupted:false
2016-05-17 21:29:47,570 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:47,571 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:47,571 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:47,571 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:47,571 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:47,571 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:47,572 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BFFC3FC0499742D0AE1F11350B15CE23
2016-05-17 21:29:47,572 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:47,572 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CCA4319F6A254CA0B22ED56520ABAE14, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:47,573 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:47,575 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@78ed3a80; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@1e9be786
2016-05-17 21:29:47,578 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:47,579 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:47,579 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520587289_0001_1_01_000000_0
2016-05-17 21:29:47,579 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520587289_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:47,580 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520587289_0001_1_01_000000_0
2016-05-17 21:29:47,580 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CCA4319F6A254CA0B22ED56520ABAE14, mem on close (mb), free: 189, total: 363, max: 672
2016-05-17 21:29:47,581 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BFFC3FC0499742D0AE1F11350B15CE23
2016-05-17 21:29:47,581 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:47,582 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:47,582 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:47,582 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:47,583 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:47,583 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:47 Completed running task attempt: attempt_1463520587289_0001_1_01_000000_0
2016-05-17 21:29:47,584 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520587289_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:47,584 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520587572, Process_Duration=8, Process_End_Time=1463520587580, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:47,585 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CCA4319F6A254CA0B22ED56520ABAE14, taskAttemptId=attempt_1463520587289_0001_1_01_000000_0, startTime=1463520587540, finishTime=1463520587584, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520587572, Process_Duration=8, Process_End_Time=1463520587580, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:47,584 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:47,585 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520587289_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:47,585 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520587289_0001_1_01_000000_0
2016-05-17 21:29:47,585 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:47,585 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:47,585 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:47,585 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520587289_0001_00_000002
2016-05-17 21:29:47,585 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:47,585 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:TASK_FINISHED]: vertexName=CCA4319F6A254CA0B22ED56520ABAE14, taskId=task_1463520587289_0001_1_01_000000, startTime=1463520587540, finishTime=1463520587585, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1463520587289_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520587572, Process_Duration=8, Process_End_Time=1463520587580, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:47,586 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520587289_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:47,586 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14] : 1
2016-05-17 21:29:47,586 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14]
2016-05-17 21:29:47,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:47,586 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:VERTEX_FINISHED]: vertexName=CCA4319F6A254CA0B22ED56520ABAE14, vertexId=vertex_1463520587289_0001_1_01, initRequestedTime=1463520587450, initedTime=1463520587457, startRequestedTime=1463520587464, startedTime=1463520587464, finishTime=1463520587586, timeTaken=122, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520587572, Process_Duration=8, Process_End_Time=1463520587580, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520587540, firstTasksToStart=[ task_1463520587289_0001_1_01_000000 ], lastTaskFinishTime=1463520587584, lastTasksToFinish=[ task_1463520587289_0001_1_01_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520587289_0001_1_01_000000 ], longestDurationTasks=[ task_1463520587289_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:47,587 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:47,587 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587289_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:47,587 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520587289_0001_00_000002, stoppedTime=1463520587587, exitStatus=0
2016-05-17 21:29:47,587 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:47,587 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587289_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:47,587 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:47,587 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520587289_0001_1
2016-05-17 21:29:47,587 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520587289_0001_1
2016-05-17 21:29:47,587 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 6EBF5AE60E3E4F7D9031C0A3C18A7954 for vertex: vertex_1463520587289_0001_1_01 [CCA4319F6A254CA0B22ED56520ABAE14]
2016-05-17 21:29:47,593 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520587289_0001_1_00 [BFFC3FC0499742D0AE1F11350B15CE23]
2016-05-17 21:29:47,594 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520587289_0001_1, queueSize=0
2016-05-17 21:29:47,594 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:47,595 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587289_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520587289_0001_1, startTime=1463520587448, finishTime=1463520587593, timeTaken=145, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=971, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762314752, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041175091, Process_Duration=9, Process_End_Time=2927041175100, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:47,595 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520587289_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:47,595 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520587289_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:47,596 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520587289_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:47,596 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520587289_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:47,596 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520587289_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:47,596 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520587289_0001_00_000002, Exiting
2016-05-17 21:29:47,596 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520587289_0001_00_000002
2016-05-17 21:29:47,596 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520587289_0001_00_000002
2016-05-17 21:29:47,596 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520587289_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:47,596 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520587289_0001_00_000002 completed successfully
2016-05-17 21:29:47 Completed Dag: dag_1463520587289_0001_1
2016-05-17 21:29:47,598 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520587289_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:47,598 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520587289_0001_1_post
2016-05-17 21:29:47,598 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:47,599 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:47,599 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587289_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:47,600 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587289_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:47,603 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/ALL_'field'_ALL, applicationId=application_1463520587289_0001
2016-05-17 21:29:47,604 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:47,604 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:47,604 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:47,604 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:47,604 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:47,605 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:47,605 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:47,605 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:47,605 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:47,605 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:47,606 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:47,606 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:47,606 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:47,610 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/7B74C835F9894FB68EEF755EF7D1B574/F5A79B0EB95F404B97D14956E904D931/.tez/application_1463520587289_0001
2016-05-17 21:29:47,610 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_ALL/_temporary
2016-05-17 21:29:47,610 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.338
2016-05-17 21:29:47,671 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_ALL
2016-05-17 21:29:47,691 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:47,692 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:47,716 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.001
2016-05-17 21:29:47,717 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 21:29:47,718 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:47,718 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:47,719 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_REPLACE
2016-05-17 21:29:47,739 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:47,739 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:47,767 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 21:29:47,772 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:29:47,772 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:47,773 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:47,773 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:47,786 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:47,790 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_SWAP
2016-05-17 21:29:47,830 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:47,830 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:47,830 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_SWAP"]
2016-05-17 21:29:47,830 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:47,830 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:47,830 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:47,831 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...platform/ALL_'field'_SWAP
2016-05-17 21:29:47,831 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:47,841 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:47,845 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:47,845 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:47,845 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:47,854 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EBEAFB1C3AE14722A859CCA2691BF736/D2F52D2B3490443C9AADD17E24CC770F/.tez/application_1463520587845_0001 doesn't exist and is created
2016-05-17 21:29:47,868 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:47,868 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:47,868 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EBEAFB1C3AE14722A859CCA2691BF736/D2F52D2B3490443C9AADD17E24CC770F/.tez/application_1463520587845_0001
2016-05-17 21:29:47,870 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520587845_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:47,879 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:47,880 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:47,880 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:47,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:47,881 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EBEAFB1C3AE14722A859CCA2691BF736/D2F52D2B3490443C9AADD17E24CC770F/.tez/application_1463520587845_0001/localmode-log-dir
2016-05-17 21:29:47,881 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EBEAFB1C3AE14722A859CCA2691BF736/D2F52D2B3490443C9AADD17E24CC770F/.tez/application_1463520587845_0001/localmode-log-dir/history.txt.appattempt_1463520587845_0001_000000, maxErrors=10
2016-05-17 21:29:47,881 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:47,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520587845_0001_000000, appSubmitTime=1463520587870, launchTime=1463520587870
2016-05-17 21:29:47,884 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:47,884 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:47,885 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:47,886 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:47,887 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:57004
2016-05-17 21:29:47,888 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:47,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520587845_0001_000000, startTime=1463520587889
2016-05-17 21:29:47,889 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:47,889 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:47,932 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:47,932 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:47,968 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:47,968 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:47,968 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_'field'_SWAP, applicationId=application_1463520587845_0001, dagName=[EBEAFB1C3AE14722A859CCA2691BF736/D2F52D2B3490443C9AADD17E24CC770F] every/(1/1) ...platform/ALL_'field'_SWAP
2016-05-17 21:29:47,981 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [EBEAFB1C3AE14722A859CCA2691BF736/D2F52D2B3490443C9AADD17E24CC770F] every/(1/1) ...platform/ALL_'field'_SWAP
2016-05-17 21:29:47,983 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520587845_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EBEAFB1C3AE14722A859CCA2691BF736/D2F52D2B3490443C9AADD17E24CC770F/.tez/application_1463520587845_0001/localmode-log-dir/dag_1463520587845_0001_1.dot
2016-05-17 21:29:47,983 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EBEAFB1C3AE14722A859CCA2691BF736/D2F52D2B3490443C9AADD17E24CC770F/.tez/application_1463520587845_0001/localmode-log-dir/dag_1463520587845_0001_1-tez-dag.pb.txt
2016-05-17 21:29:47,989 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520587845_0001_1
2016-05-17 21:29:47,989 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:47,989 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [EBEAFB1C3AE14722A859CCA2691BF736/D2F52D2B3490443C9AADD17E24CC770F] every/(1/1) ...platform/ALL_'field'_SWAP
2016-05-17 21:29:47 Running Dag: dag_1463520587845_0001_1
2016-05-17 21:29:47,999 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:47,999 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520587845_0001_1, submitTime=1463520587981
2016-05-17 21:29:48,000 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C27E59E4A32C4A72A861BA86BFBF4142
2016-05-17 21:29:48,000 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:48,000 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:48,000 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520587845_0001_1, initTime=1463520587999
2016-05-17 21:29:48,001 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520587845_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:48,001 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:DAG_STARTED]: dagID=dag_1463520587845_0001_1, startTime=1463520588001
2016-05-17 21:29:48,001 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:48,001 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520587845_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:48,001 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D9547CBEE096441484A1EA6E411F0637 : {BFB65EA60CEE4B0299DFD1740CA692DE={InputName=BFB65EA60CEE4B0299DFD1740CA692DE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:48,001 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: BFB65EA60CEE4B0299DFD1740CA692DE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:48,001 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637]
2016-05-17 21:29:48,002 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637]
2016-05-17 21:29:48,002 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637]
2016-05-17 21:29:48,002 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637]. Starting root input initializers: 1
2016-05-17 21:29:48,002 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:48,002 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:48,002 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142]
2016-05-17 21:29:48,003 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_'field'_SWAP, applicationId=application_1463520587845_0001, dagName=[EBEAFB1C3AE14722A859CCA2691BF736/D2F52D2B3490443C9AADD17E24CC770F] every/(1/1) ...platform/ALL_'field'_SWAP
2016-05-17 21:29:48,003 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: BFB65EA60CEE4B0299DFD1740CA692DE on vertex vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637]
2016-05-17 21:29:48,005 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:48,005 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input BFB65EA60CEE4B0299DFD1740CA692DE asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:48,005 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:48,005 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142]
2016-05-17 21:29:48,005 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142]
2016-05-17 21:29:48,006 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142]
2016-05-17 21:29:48,006 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=62C966B0D8AC433E818E8522EC3C63AC, vertexId=vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:48,006 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=62C966B0D8AC433E818E8522EC3C63AC, vertexId=vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142]
2016-05-17 21:29:48,007 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:48,007 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:48,007 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=62C966B0D8AC433E818E8522EC3C63AC, vertexName=C27E59E4A32C4A72A861BA86BFBF4142, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:48,008 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=62C966B0D8AC433E818E8522EC3C63AC, vertexId=vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142]
2016-05-17 21:29:48,009 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:48,009 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:48,009 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C27E59E4A32C4A72A861BA86BFBF4142, vertexId=vertex_1463520587845_0001_1_01, initRequestedTime=1463520588002, initedTime=1463520588009, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:48,009 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:48,010 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: BFB65EA60CEE4B0299DFD1740CA692DE on vertex vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637]
2016-05-17 21:29:48,010 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:48,010 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {BFB65EA60CEE4B0299DFD1740CA692DE=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:48,011 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637] parallelism set to 1
2016-05-17 21:29:48,011 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:48,011 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520587845_0001_1_00, eventCount=1
2016-05-17 21:29:48,012 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:48,012 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D9547CBEE096441484A1EA6E411F0637, vertexId=vertex_1463520587845_0001_1_00, initRequestedTime=1463520588001, initedTime=1463520588011, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:48,012 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637] with distanceFromRoot: 0
2016-05-17 21:29:48,012 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:48,012 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D9547CBEE096441484A1EA6E411F0637
2016-05-17 21:29:48,012 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637]
2016-05-17 21:29:48,013 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:48,013 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520587845_0001_1_00, startRequestedTime=1463520588012, startedTime=1463520588012
2016-05-17 21:29:48,013 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:48,014 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:48,014 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:TASK_STARTED]: vertexName=D9547CBEE096441484A1EA6E411F0637, taskId=task_1463520587845_0001_1_00_000000, scheduledTime=1463520588013, launchTime=1463520588013
2016-05-17 21:29:48,014 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520587845_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:48,014 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520587845_0001_1_00 for vertex: vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142] numStartedSources: 1 numSources: 1
2016-05-17 21:29:48,014 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142] with distanceFromRoot: 1
2016-05-17 21:29:48,015 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637]
2016-05-17 21:29:48,015 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D9547CBEE096441484A1EA6E411F0637 in vertex: C27E59E4A32C4A72A861BA86BFBF4142
2016-05-17 21:29:48,015 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C27E59E4A32C4A72A861BA86BFBF4142 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:48,015 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:48,016 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520587845_0001_1_01, startRequestedTime=1463520588015, startedTime=1463520588015
2016-05-17 21:29:48,016 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:48,016 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [EBEAFB1C3AE14722A859CCA2691BF736/D2F52D2B3490443C9AADD17E24CC770F] every/(1/1) ...platform/ALL_'field'_SWAP, VertexName: D9547CBEE096441484A1EA6E411F0637, VertexParallelism: 1, TaskAttemptID:attempt_1463520587845_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BFB65EA60CEE4B0299DFD1740CA692DE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C27E59E4A32C4A72A861BA86BFBF4142, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:48,016 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520587845_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:48,017 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:48,017 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:48,018 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:48,018 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:48,018 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520587845_0001_00_000001
2016-05-17 21:29:48,018 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587845_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:48,019 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520587845_0001, with dag id: dag_1463520587845_0001_1
2016-05-17 21:29:48,019 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520587845_0001_00_000001
2016-05-17 21:29:48,019 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:48,021 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587845_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:48,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:48,021 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520587845_0001_00_000001, launchTime=1463520588020
2016-05-17 21:29:48,021 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520587845_0001_00_000001
2016-05-17 21:29:48,022 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520587845_0001_1_00_000000_0] to container: [container_1463520587845_0001_00_000001]
2016-05-17 21:29:48,022 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587845_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:48,022 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520587845_0001_00_000001 given task: attempt_1463520587845_0001_1_00_000000_0
2016-05-17 21:29:48,022 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520587845_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520587845_0001_1_00_000000_0
2016-05-17 21:29:48,022 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520587845_0001_1_00_000000_0] started. Is using containerId: [container_1463520587845_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:48 Starting to run new task attempt: attempt_1463520587845_0001_1_00_000000_0
2016-05-17 21:29:48,023 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D9547CBEE096441484A1EA6E411F0637, taskAttemptId=attempt_1463520587845_0001_1_00_000000_0, startTime=1463520588022, containerId=container_1463520587845_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520587845_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:48,023 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520587845_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:48,023 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520587845_0001_1_00_000000_0
2016-05-17 21:29:48,023 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520587845_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:48,023 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:48,023 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:48,023 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:48,023 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:48,023 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:48,023 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:48,024 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:48,024 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EBEAFB1C3AE14722A859CCA2691BF736/D2F52D2B3490443C9AADD17E24CC770F] every/(1/1) ...platform/ALL_'field'_SWAP, VertexName: D9547CBEE096441484A1EA6E411F0637, VertexParallelism: 1, TaskAttemptID:attempt_1463520587845_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BFB65EA60CEE4B0299DFD1740CA692DE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C27E59E4A32C4A72A861BA86BFBF4142, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:48,024 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:48,024 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520587845_0001_1_00_000000_0
2016-05-17 21:29:48,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:48,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:48,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BFB65EA60CEE4B0299DFD1740CA692DE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:48,025 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520587845_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:48,026 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:48,026 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BFB65EA60CEE4B0299DFD1740CA692DE
2016-05-17 21:29:48,026 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: BFB65EA60CEE4B0299DFD1740CA692DE
2016-05-17 21:29:48,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:48,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C27E59E4A32C4A72A861BA86BFBF4142, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:48,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:48,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C27E59E4A32C4A72A861BA86BFBF4142
2016-05-17 21:29:48,029 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:48,029 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:48,029 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C27E59E4A32C4A72A861BA86BFBF4142
2016-05-17 21:29:48,029 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C27E59E4A32C4A72A861BA86BFBF4142
2016-05-17 21:29:48,030 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:48,030 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: BFB65EA60CEE4B0299DFD1740CA692DE
2016-05-17 21:29:48,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BFB65EA60CEE4B0299DFD1740CA692DE
2016-05-17 21:29:48,032 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D9547CBEE096441484A1EA6E411F0637, ordinal: 0
2016-05-17 21:29:48,032 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D9547CBEE096441484A1EA6E411F0637, mem on start (mb), free: 85, total: 363, max: 672
2016-05-17 21:29:48,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:48,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:48,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:48,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:48,032 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:48,033 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:48,033 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:48,033 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:48,033 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:48,033 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:48,033 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BFB65EA60CEE4B0299DFD1740CA692DE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:48,033 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C27E59E4A32C4A72A861BA86BFBF4142, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:48,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:48,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BFB65EA60CEE4B0299DFD1740CA692DE being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:48,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BFB65EA60CEE4B0299DFD1740CA692DE
2016-05-17 21:29:48,034 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:48,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BFB65EA60CEE4B0299DFD1740CA692DE
2016-05-17 21:29:48,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:48,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:48,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:48,034 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520587845_0001_1_00_000000_0
2016-05-17 21:29:48,036 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:48,037 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:48,037 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: BFB65EA60CEE4B0299DFD1740CA692DE
2016-05-17 21:29:48,037 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 4D5D38228D754E379C82B7148EA826DB
2016-05-17 21:29:48,037 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 4D5D38228D754E379C82B7148EA826DB
2016-05-17 21:29:48,037 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@35d73cb2
2016-05-17 21:29:48,038 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:48,038 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@35d73cb2
2016-05-17 21:29:48,038 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:48,038 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:48,040 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:48,040 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@57417bcc; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@c26bad5; comparator=cascading.tuple.hadoop.util.TupleComparator@42b3fc2e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:48,040 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:48,040 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:48,040 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:48,062 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:48,062 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:48,062 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:48,063 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:48,063 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:48,064 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:48,064 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:48,064 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D9547CBEE096441484A1EA6E411F0637, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:48,066 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520587845_0001_1_00_000000_0
2016-05-17 21:29:48,067 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D9547CBEE096441484A1EA6E411F0637, mem on close (mb), free: 195, total: 363, max: 672
2016-05-17 21:29:48,067 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:48,067 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:48,067 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:48,067 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:48,070 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:48 Completed running task attempt: attempt_1463520587845_0001_1_00_000000_0
2016-05-17 21:29:48,072 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520587845_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:48,073 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520587845_0001_1_00, eventCount=1
2016-05-17 21:29:48,073 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520588064, Process_Duration=2, Process_End_Time=1463520588066, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:48,073 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:48,073 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:48,073 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D9547CBEE096441484A1EA6E411F0637, taskAttemptId=attempt_1463520587845_0001_1_00_000000_0, startTime=1463520588022, finishTime=1463520588073, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520588064, Process_Duration=2, Process_End_Time=1463520588066, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:48,073 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520587845_0001_00_000001
2016-05-17 21:29:48,073 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520587845_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:48,073 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520587845_0001_1_00_000000_0
2016-05-17 21:29:48,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:TASK_FINISHED]: vertexName=D9547CBEE096441484A1EA6E411F0637, taskId=task_1463520587845_0001_1_00_000000, startTime=1463520588022, finishTime=1463520588073, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1463520587845_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520588064, Process_Duration=2, Process_End_Time=1463520588066, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:48,074 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520587845_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:48,074 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637] : 1
2016-05-17 21:29:48,074 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637]
2016-05-17 21:29:48,073 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:48,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:VERTEX_FINISHED]: vertexName=D9547CBEE096441484A1EA6E411F0637, vertexId=vertex_1463520587845_0001_1_00, initRequestedTime=1463520588001, initedTime=1463520588011, startRequestedTime=1463520588012, startedTime=1463520588012, finishTime=1463520588074, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520588064, Process_Duration=2, Process_End_Time=1463520588066, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520588022, firstTasksToStart=[ task_1463520587845_0001_1_00_000000 ], lastTaskFinishTime=1463520588073, lastTasksToFinish=[ task_1463520587845_0001_1_00_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520587845_0001_1_00_000000 ], longestDurationTasks=[ task_1463520587845_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:48,074 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:48,074 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142] attempt: attempt_1463520587845_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:48,074 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:48,075 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:48,073 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:48,074 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C27E59E4A32C4A72A861BA86BFBF4142
2016-05-17 21:29:48,076 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142]
2016-05-17 21:29:48,076 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:48,076 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587845_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:48,076 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520587845_0001_00_000001, stoppedTime=1463520588076, exitStatus=0
2016-05-17 21:29:48,076 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587845_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:48,076 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:TASK_STARTED]: vertexName=C27E59E4A32C4A72A861BA86BFBF4142, taskId=task_1463520587845_0001_1_01_000000, scheduledTime=1463520588076, launchTime=1463520588076
2016-05-17 21:29:48,077 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520587845_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:48,077 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142]
2016-05-17 21:29:48,077 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520587845_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:48,077 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [EBEAFB1C3AE14722A859CCA2691BF736/D2F52D2B3490443C9AADD17E24CC770F] every/(1/1) ...platform/ALL_'field'_SWAP, VertexName: C27E59E4A32C4A72A861BA86BFBF4142, VertexParallelism: 1, TaskAttemptID:attempt_1463520587845_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D9547CBEE096441484A1EA6E411F0637, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=62C966B0D8AC433E818E8522EC3C63AC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:48,076 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:48,077 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:48,077 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520587845_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:48,077 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587845_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:48,078 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:48,078 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520587845_0001_00_000002
2016-05-17 21:29:48,078 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587845_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:48,080 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520587845_0001_00_000002
2016-05-17 21:29:48,080 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:48,081 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587845_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:48,081 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:48,081 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520587845_0001_00_000002, launchTime=1463520588081
2016-05-17 21:29:48,082 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520587845_0001_00_000002
2016-05-17 21:29:48,082 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520587845_0001_1_01_000000_0] to container: [container_1463520587845_0001_00_000002]
2016-05-17 21:29:48,082 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587845_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:48,083 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520587845_0001_00_000002 given task: attempt_1463520587845_0001_1_01_000000_0
2016-05-17 21:29:48,083 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520587845_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520587845_0001_1_01_000000_0
2016-05-17 21:29:48,083 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520587845_0001_1_01_000000_0] started. Is using containerId: [container_1463520587845_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:48 Starting to run new task attempt: attempt_1463520587845_0001_1_01_000000_0
2016-05-17 21:29:48,083 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520587845_0001_1_01_000000_0
2016-05-17 21:29:48,083 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:48,083 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:48,083 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C27E59E4A32C4A72A861BA86BFBF4142, taskAttemptId=attempt_1463520587845_0001_1_01_000000_0, startTime=1463520588083, containerId=container_1463520587845_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520587845_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:48,083 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520587845_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:48,083 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520587845_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:48,084 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:48,084 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:48,084 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:48,084 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:48,084 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:48,085 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EBEAFB1C3AE14722A859CCA2691BF736/D2F52D2B3490443C9AADD17E24CC770F] every/(1/1) ...platform/ALL_'field'_SWAP, VertexName: C27E59E4A32C4A72A861BA86BFBF4142, VertexParallelism: 1, TaskAttemptID:attempt_1463520587845_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D9547CBEE096441484A1EA6E411F0637, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=62C966B0D8AC433E818E8522EC3C63AC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:48,085 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520587845_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:48,085 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520587845_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:48,085 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520587845_0001_00_000001, Exiting
2016-05-17 21:29:48,085 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520587845_0001_00_000001
2016-05-17 21:29:48,085 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520587845_0001_00_000001
2016-05-17 21:29:48,086 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:48,086 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520587845_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:48,086 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520587845_0001_00_000001 completed successfully
2016-05-17 21:29:48,086 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587845_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:48,086 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520587845_0001_1_01_000000_0
2016-05-17 21:29:48,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:48,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:48,087 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520587845_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:48,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D9547CBEE096441484A1EA6E411F0637, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:48,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=62C966B0D8AC433E818E8522EC3C63AC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:48,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:48,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:48,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:48,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D9547CBEE096441484A1EA6E411F0637
2016-05-17 21:29:48,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 62C966B0D8AC433E818E8522EC3C63AC
2016-05-17 21:29:48,089 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:48,089 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 62C966B0D8AC433E818E8522EC3C63AC
2016-05-17 21:29:48,090 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:48,090 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:48,090 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D9547CBEE096441484A1EA6E411F0637
2016-05-17 21:29:48,090 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D9547CBEE096441484A1EA6E411F0637
2016-05-17 21:29:48,092 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:48,097 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:48,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 62C966B0D8AC433E818E8522EC3C63AC
2016-05-17 21:29:48,097 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C27E59E4A32C4A72A861BA86BFBF4142, ordinal: 1
2016-05-17 21:29:48,097 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C27E59E4A32C4A72A861BA86BFBF4142, mem on start (mb), free: 189, total: 363, max: 672
2016-05-17 21:29:48,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:48,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:48,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:48,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:48,098 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:48,099 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:48,099 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:48,099 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:48,099 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:48,099 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:48,099 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 62C966B0D8AC433E818E8522EC3C63AC, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:48,099 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D9547CBEE096441484A1EA6E411F0637, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:48,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:48,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D9547CBEE096441484A1EA6E411F0637
2016-05-17 21:29:48,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D9547CBEE096441484A1EA6E411F0637 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:48,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:48,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:48,100 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:48,100 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:48,101 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D9547CBEE096441484A1EA6E411F0637 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:48,101 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:48,101 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:48,101 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:48,102 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D9547CBEE096441484A1EA6E411F0637
2016-05-17 21:29:48,102 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:48,102 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520587845_0001_1_01_000000_0
2016-05-17 21:29:48,102 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520587845_0001_1_00_000000_0_10255, runDuration: 43453, ]
2016-05-17 21:29:48,104 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520587845_0001_1_00_000000_0_10255] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520588104, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:48,104 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D9547CBEE096441484A1EA6E411F0637
2016-05-17 21:29:48,104 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:48,104 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D9547CBEE096441484A1EA6E411F0637] #43 in 1ms
2016-05-17 21:29:48,104 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D9547CBEE096441484A1EA6E411F0637] #43, status:false, isInterrupted:false
2016-05-17 21:29:48,105 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:48,105 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:48,106 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 4D5D38228D754E379C82B7148EA826DB
2016-05-17 21:29:48,106 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_SWAP"], id: 62C966B0D8AC433E818E8522EC3C63AC
2016-05-17 21:29:48,106 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_SWAP"]
2016-05-17 21:29:48,106 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 4D5D38228D754E379C82B7148EA826DB, for 1 inputs
2016-05-17 21:29:48,106 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:48,106 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:48,106 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:48,106 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:48,107 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:48,107 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D9547CBEE096441484A1EA6E411F0637
2016-05-17 21:29:48,107 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:48,107 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C27E59E4A32C4A72A861BA86BFBF4142, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:29:48,108 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:48,109 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@3dc81f47; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@1d7d8e01
2016-05-17 21:29:48,112 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:48,112 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:48,112 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520587845_0001_1_01_000000_0
2016-05-17 21:29:48,112 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520587845_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:48,113 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520587845_0001_1_01_000000_0
2016-05-17 21:29:48,113 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C27E59E4A32C4A72A861BA86BFBF4142, mem on close (mb), free: 187, total: 363, max: 672
2016-05-17 21:29:48,113 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D9547CBEE096441484A1EA6E411F0637
2016-05-17 21:29:48,113 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:48,114 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:48,114 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:48,114 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:48,114 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:48,115 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:48 Completed running task attempt: attempt_1463520587845_0001_1_01_000000_0
2016-05-17 21:29:48,115 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520587845_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:48,116 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:48,116 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C27E59E4A32C4A72A861BA86BFBF4142, taskAttemptId=attempt_1463520587845_0001_1_01_000000_0, startTime=1463520588083, finishTime=1463520588116, timeTaken=33, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520588107, Process_Duration=6, Process_End_Time=1463520588113, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:48,116 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520587845_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:48,116 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520587845_0001_1_01_000000_0
2016-05-17 21:29:48,116 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:48,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:TASK_FINISHED]: vertexName=C27E59E4A32C4A72A861BA86BFBF4142, taskId=task_1463520587845_0001_1_01_000000, startTime=1463520588083, finishTime=1463520588116, timeTaken=33, status=SUCCEEDED, successfulAttemptID=attempt_1463520587845_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520588107, Process_Duration=6, Process_End_Time=1463520588113, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:48,117 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:48,117 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520587845_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:48,117 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142] : 1
2016-05-17 21:29:48,117 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142]
2016-05-17 21:29:48,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:48,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:VERTEX_FINISHED]: vertexName=C27E59E4A32C4A72A861BA86BFBF4142, vertexId=vertex_1463520587845_0001_1_01, initRequestedTime=1463520588002, initedTime=1463520588009, startRequestedTime=1463520588015, startedTime=1463520588015, finishTime=1463520588117, timeTaken=102, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520588107, Process_Duration=6, Process_End_Time=1463520588113, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520588083, firstTasksToStart=[ task_1463520587845_0001_1_01_000000 ], lastTaskFinishTime=1463520588116, lastTasksToFinish=[ task_1463520587845_0001_1_01_000000 ], minTaskDuration=33, maxTaskDuration=33, avgTaskDuration=33.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520587845_0001_1_01_000000 ], longestDurationTasks=[ task_1463520587845_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:48,118 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:48,118 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587845_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:48,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520587845_0001_00_000002, stoppedTime=1463520588118, exitStatus=0
2016-05-17 21:29:48,118 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587845_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:48,119 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:48,119 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520587845_0001_1
2016-05-17 21:29:48,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:48,119 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520587845_0001_1
2016-05-17 21:29:48,119 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520587845_0001_1_00 [D9547CBEE096441484A1EA6E411F0637]
2016-05-17 21:29:48,119 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 62C966B0D8AC433E818E8522EC3C63AC for vertex: vertex_1463520587845_0001_1_01 [C27E59E4A32C4A72A861BA86BFBF4142]
2016-05-17 21:29:48,120 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520588107, Process_Duration=6, Process_End_Time=1463520588113, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:48,120 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:48,121 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:48,121 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520587845_0001_00_000002
2016-05-17 21:29:48,121 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520587845_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:48,121 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520587845_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:48,122 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520587845_0001_00_000002, Exiting
2016-05-17 21:29:48,122 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520587845_0001_00_000002
2016-05-17 21:29:48,122 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520587845_0001_00_000002
2016-05-17 21:29:48,122 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520587845_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:48,122 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520587845_0001_00_000002 completed successfully
2016-05-17 21:29:48,127 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520587845_0001_1, queueSize=0
2016-05-17 21:29:48,127 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:48,128 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520587845_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520587845_0001_1, startTime=1463520588001, finishTime=1463520588126, timeTaken=125, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=345, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762314752, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041176171, Process_Duration=8, Process_End_Time=2927041176179, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:48,128 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520587845_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:48,128 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520587845_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:48,128 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520587845_0001_00_000002
2016-05-17 21:29:48,129 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520587845_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:48 Completed Dag: dag_1463520587845_0001_1
2016-05-17 21:29:48,129 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520587845_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:48,130 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520587845_0001_1_post
2016-05-17 21:29:48,130 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:48,130 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:48,134 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_'field'_SWAP, applicationId=application_1463520587845_0001
2016-05-17 21:29:48,135 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:48,135 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:48,135 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:48,135 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:48,136 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:48,136 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:48,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:48,136 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:48,136 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:48,137 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:48,137 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:48,137 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:48,137 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:48,141 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/EBEAFB1C3AE14722A859CCA2691BF736/D2F52D2B3490443C9AADD17E24CC770F/.tez/application_1463520587845_0001
2016-05-17 21:29:48,141 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_SWAP/_temporary
2016-05-17 21:29:48,141 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.311
2016-05-17 21:29:48,205 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_SWAP
2016-05-17 21:29:48,224 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:48,224 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:48,250 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 21:29:48,254 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:29:48,254 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:48,255 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:48,255 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:48,266 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:48,270 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_-1
2016-05-17 21:29:48,308 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:48,308 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:48,308 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_-1"]
2016-05-17 21:29:48,309 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:48,309 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:48,309 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:48,309 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-05-17 21:29:48,309 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:48,320 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:48,324 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:48,325 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:48,325 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:48,333 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/524C7ECDD6C545DDA1D0749C2BFFA3B0/F6D7065FFF8E45A18A5A5254A8720570/.tez/application_1463520588324_0001 doesn't exist and is created
2016-05-17 21:29:48,348 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:48,348 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:48,348 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/524C7ECDD6C545DDA1D0749C2BFFA3B0/F6D7065FFF8E45A18A5A5254A8720570/.tez/application_1463520588324_0001
2016-05-17 21:29:48,350 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520588324_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:48,359 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:48,359 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:48,360 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:48,360 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:48,360 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/524C7ECDD6C545DDA1D0749C2BFFA3B0/F6D7065FFF8E45A18A5A5254A8720570/.tez/application_1463520588324_0001/localmode-log-dir
2016-05-17 21:29:48,360 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/524C7ECDD6C545DDA1D0749C2BFFA3B0/F6D7065FFF8E45A18A5A5254A8720570/.tez/application_1463520588324_0001/localmode-log-dir/history.txt.appattempt_1463520588324_0001_000000, maxErrors=10
2016-05-17 21:29:48,360 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:48,361 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520588324_0001_000000, appSubmitTime=1463520588350, launchTime=1463520588350
2016-05-17 21:29:48,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:48,364 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:48,365 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:48,365 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:48,366 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:43692
2016-05-17 21:29:48,367 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:48,368 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520588324_0001_000000, startTime=1463520588368
2016-05-17 21:29:48,368 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:48,368 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:48,388 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:48,388 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:48,448 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:48,448 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:48,448 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_UNKNOWN_-1, applicationId=application_1463520588324_0001, dagName=[524C7ECDD6C545DDA1D0749C2BFFA3B0/F6D7065FFF8E45A18A5A5254A8720570] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-05-17 21:29:48,461 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [524C7ECDD6C545DDA1D0749C2BFFA3B0/F6D7065FFF8E45A18A5A5254A8720570] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-05-17 21:29:48,462 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520588324_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/524C7ECDD6C545DDA1D0749C2BFFA3B0/F6D7065FFF8E45A18A5A5254A8720570/.tez/application_1463520588324_0001/localmode-log-dir/dag_1463520588324_0001_1.dot
2016-05-17 21:29:48,463 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/524C7ECDD6C545DDA1D0749C2BFFA3B0/F6D7065FFF8E45A18A5A5254A8720570/.tez/application_1463520588324_0001/localmode-log-dir/dag_1463520588324_0001_1-tez-dag.pb.txt
2016-05-17 21:29:48,468 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520588324_0001_1
2016-05-17 21:29:48,468 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:48,468 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [524C7ECDD6C545DDA1D0749C2BFFA3B0/F6D7065FFF8E45A18A5A5254A8720570] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-05-17 21:29:48 Running Dag: dag_1463520588324_0001_1
2016-05-17 21:29:48,480 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520588324_0001_1, submitTime=1463520588461
2016-05-17 21:29:48,480 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:48,480 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E5F379DA932B440180AFDCD07DC9CF74
2016-05-17 21:29:48,480 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:48,480 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:48,481 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520588324_0001_1, initTime=1463520588480
2016-05-17 21:29:48,481 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520588324_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:48,481 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:DAG_STARTED]: dagID=dag_1463520588324_0001_1, startTime=1463520588481
2016-05-17 21:29:48,481 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:48,481 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520588324_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:48,481 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:48,481 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E98EC7CB8EA046A4BF4123D30A1D6778 : {0A3F111F562F468D86ED19E1A5ABCC2F={InputName=0A3F111F562F468D86ED19E1A5ABCC2F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:48,481 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0A3F111F562F468D86ED19E1A5ABCC2F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:48,482 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778]
2016-05-17 21:29:48,482 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778]
2016-05-17 21:29:48,482 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778]
2016-05-17 21:29:48,482 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_UNKNOWN_-1, applicationId=application_1463520588324_0001, dagName=[524C7ECDD6C545DDA1D0749C2BFFA3B0/F6D7065FFF8E45A18A5A5254A8720570] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-05-17 21:29:48,482 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778]. Starting root input initializers: 1
2016-05-17 21:29:48,482 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:48,483 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74]
2016-05-17 21:29:48,485 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0A3F111F562F468D86ED19E1A5ABCC2F on vertex vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778]
2016-05-17 21:29:48,485 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:48,485 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74]
2016-05-17 21:29:48,486 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74]
2016-05-17 21:29:48,486 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74]
2016-05-17 21:29:48,486 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=AC4E926CEF6B42AF82C220377FAAAF72, vertexId=vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:48,486 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=AC4E926CEF6B42AF82C220377FAAAF72, vertexId=vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74]
2016-05-17 21:29:48,488 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:48,488 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=AC4E926CEF6B42AF82C220377FAAAF72, vertexName=E5F379DA932B440180AFDCD07DC9CF74, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:48,488 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=AC4E926CEF6B42AF82C220377FAAAF72, vertexId=vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74]
2016-05-17 21:29:48,488 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0A3F111F562F468D86ED19E1A5ABCC2F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:48,489 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:48,489 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:48,489 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:48,489 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E5F379DA932B440180AFDCD07DC9CF74, vertexId=vertex_1463520588324_0001_1_01, initRequestedTime=1463520588483, initedTime=1463520588489, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:48,490 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:48,491 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:48,491 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0A3F111F562F468D86ED19E1A5ABCC2F on vertex vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778]
2016-05-17 21:29:48,492 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:48,492 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0A3F111F562F468D86ED19E1A5ABCC2F=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:48,492 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778] parallelism set to 1
2016-05-17 21:29:48,492 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:48,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520588324_0001_1_00, eventCount=1
2016-05-17 21:29:48,493 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:48,493 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E98EC7CB8EA046A4BF4123D30A1D6778, vertexId=vertex_1463520588324_0001_1_00, initRequestedTime=1463520588481, initedTime=1463520588493, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:48,493 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778] with distanceFromRoot: 0
2016-05-17 21:29:48,493 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:48,493 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E98EC7CB8EA046A4BF4123D30A1D6778
2016-05-17 21:29:48,494 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778]
2016-05-17 21:29:48,494 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:48,494 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520588324_0001_1_00, startRequestedTime=1463520588493, startedTime=1463520588493
2016-05-17 21:29:48,494 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:48,495 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:48,495 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:TASK_STARTED]: vertexName=E98EC7CB8EA046A4BF4123D30A1D6778, taskId=task_1463520588324_0001_1_00_000000, scheduledTime=1463520588495, launchTime=1463520588495
2016-05-17 21:29:48,495 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520588324_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:48,495 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520588324_0001_1_00 for vertex: vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74] numStartedSources: 1 numSources: 1
2016-05-17 21:29:48,496 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74] with distanceFromRoot: 1
2016-05-17 21:29:48,496 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778]
2016-05-17 21:29:48,496 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E98EC7CB8EA046A4BF4123D30A1D6778 in vertex: E5F379DA932B440180AFDCD07DC9CF74
2016-05-17 21:29:48,496 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E5F379DA932B440180AFDCD07DC9CF74 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:48,497 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:48,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520588324_0001_1_01, startRequestedTime=1463520588496, startedTime=1463520588496
2016-05-17 21:29:48,497 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:48,497 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [524C7ECDD6C545DDA1D0749C2BFFA3B0/F6D7065FFF8E45A18A5A5254A8720570] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1, VertexName: E98EC7CB8EA046A4BF4123D30A1D6778, VertexParallelism: 1, TaskAttemptID:attempt_1463520588324_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0A3F111F562F468D86ED19E1A5ABCC2F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E5F379DA932B440180AFDCD07DC9CF74, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:48,498 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520588324_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:48,498 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:48,499 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520588324_0001, with dag id: dag_1463520588324_0001_1
2016-05-17 21:29:48,499 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:48,499 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:48,499 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:48,500 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520588324_0001_00_000001
2016-05-17 21:29:48,500 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588324_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:48,501 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520588324_0001_00_000001
2016-05-17 21:29:48,501 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:48,502 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588324_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:48,502 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:48,502 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520588324_0001_00_000001, launchTime=1463520588502
2016-05-17 21:29:48,503 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520588324_0001_00_000001
2016-05-17 21:29:48,504 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520588324_0001_1_00_000000_0] to container: [container_1463520588324_0001_00_000001]
2016-05-17 21:29:48,504 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588324_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:48,504 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520588324_0001_00_000001 given task: attempt_1463520588324_0001_1_00_000000_0
2016-05-17 21:29:48,505 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520588324_0001_1_00_000000_0] started. Is using containerId: [container_1463520588324_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:48,505 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E98EC7CB8EA046A4BF4123D30A1D6778, taskAttemptId=attempt_1463520588324_0001_1_00_000000_0, startTime=1463520588504, containerId=container_1463520588324_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520588324_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:48,505 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520588324_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:48,505 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520588324_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:48,505 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:48,505 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520588324_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520588324_0001_1_00_000000_0
2016-05-17 21:29:48 Starting to run new task attempt: attempt_1463520588324_0001_1_00_000000_0
2016-05-17 21:29:48,506 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520588324_0001_1_00_000000_0
2016-05-17 21:29:48,506 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:48,506 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:48,506 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:48,506 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:48,507 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:48,507 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:48,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [524C7ECDD6C545DDA1D0749C2BFFA3B0/F6D7065FFF8E45A18A5A5254A8720570] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1, VertexName: E98EC7CB8EA046A4BF4123D30A1D6778, VertexParallelism: 1, TaskAttemptID:attempt_1463520588324_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0A3F111F562F468D86ED19E1A5ABCC2F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E5F379DA932B440180AFDCD07DC9CF74, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:48,507 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:48,507 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520588324_0001_1_00_000000_0
2016-05-17 21:29:48,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:48,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:48,508 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520588324_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:48,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:48,510 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:48,510 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E5F379DA932B440180AFDCD07DC9CF74, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:48,511 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:48,511 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E5F379DA932B440180AFDCD07DC9CF74
2016-05-17 21:29:48,510 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0A3F111F562F468D86ED19E1A5ABCC2F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:48,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:48,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0A3F111F562F468D86ED19E1A5ABCC2F
2016-05-17 21:29:48,512 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0A3F111F562F468D86ED19E1A5ABCC2F
2016-05-17 21:29:48,512 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:48,513 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E5F379DA932B440180AFDCD07DC9CF74
2016-05-17 21:29:48,513 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E5F379DA932B440180AFDCD07DC9CF74
2016-05-17 21:29:48,514 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E98EC7CB8EA046A4BF4123D30A1D6778, ordinal: 0
2016-05-17 21:29:48,515 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E98EC7CB8EA046A4BF4123D30A1D6778, mem on start (mb), free: 97, total: 363, max: 672
2016-05-17 21:29:48,515 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:48,515 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:48,516 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:48,516 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0A3F111F562F468D86ED19E1A5ABCC2F
2016-05-17 21:29:48,516 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0A3F111F562F468D86ED19E1A5ABCC2F
2016-05-17 21:29:48,516 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:48,516 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:48,517 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:48,517 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:48,517 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:48,517 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:48,517 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:48,518 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:48,518 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0A3F111F562F468D86ED19E1A5ABCC2F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:48,518 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E5F379DA932B440180AFDCD07DC9CF74, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:48,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:48,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0A3F111F562F468D86ED19E1A5ABCC2F being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:48,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0A3F111F562F468D86ED19E1A5ABCC2F
2016-05-17 21:29:48,519 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:48,519 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0A3F111F562F468D86ED19E1A5ABCC2F
2016-05-17 21:29:48,519 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:48,519 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:48,519 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:48,519 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520588324_0001_1_00_000000_0
2016-05-17 21:29:48,519 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:48,520 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@329d3e8
2016-05-17 21:29:48,521 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@329d3e8
2016-05-17 21:29:48,521 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:48,521 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:48,522 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:48,523 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0A3F111F562F468D86ED19E1A5ABCC2F
2016-05-17 21:29:48,523 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C897BB870D2A45A0A084C6DA3E1C356C
2016-05-17 21:29:48,523 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C897BB870D2A45A0A084C6DA3E1C356C
2016-05-17 21:29:48,525 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:48,529 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:48,529 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@55ce23d4; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@50925f4e; comparator=cascading.tuple.hadoop.util.TupleComparator@39af2335; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:48,529 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:48,529 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:48,529 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:48,550 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:48,550 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:48,550 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:48,550 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:48,550 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:48,551 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:48,551 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:48,551 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E98EC7CB8EA046A4BF4123D30A1D6778, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:48,553 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520588324_0001_1_00_000000_0
2016-05-17 21:29:48,554 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E98EC7CB8EA046A4BF4123D30A1D6778, mem on close (mb), free: 194, total: 363, max: 672
2016-05-17 21:29:48,554 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:48,554 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:48,554 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:48,554 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:48,557 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:48 Completed running task attempt: attempt_1463520588324_0001_1_00_000000_0
2016-05-17 21:29:48,559 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520588324_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:48,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520588551, Process_Duration=2, Process_End_Time=1463520588553, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:48,559 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520588324_0001_1_00, eventCount=1
2016-05-17 21:29:48,560 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E98EC7CB8EA046A4BF4123D30A1D6778, taskAttemptId=attempt_1463520588324_0001_1_00_000000_0, startTime=1463520588504, finishTime=1463520588560, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520588551, Process_Duration=2, Process_End_Time=1463520588553, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:48,560 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520588324_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:48,560 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520588324_0001_1_00_000000_0
2016-05-17 21:29:48,561 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:TASK_FINISHED]: vertexName=E98EC7CB8EA046A4BF4123D30A1D6778, taskId=task_1463520588324_0001_1_00_000000, startTime=1463520588504, finishTime=1463520588560, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1463520588324_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520588551, Process_Duration=2, Process_End_Time=1463520588553, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:48,561 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520588324_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:48,561 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778] : 1
2016-05-17 21:29:48,561 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778]
2016-05-17 21:29:48,562 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:VERTEX_FINISHED]: vertexName=E98EC7CB8EA046A4BF4123D30A1D6778, vertexId=vertex_1463520588324_0001_1_00, initRequestedTime=1463520588481, initedTime=1463520588493, startRequestedTime=1463520588493, startedTime=1463520588493, finishTime=1463520588561, timeTaken=68, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520588551, Process_Duration=2, Process_End_Time=1463520588553, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520588504, firstTasksToStart=[ task_1463520588324_0001_1_00_000000 ], lastTaskFinishTime=1463520588560, lastTasksToFinish=[ task_1463520588324_0001_1_00_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520588324_0001_1_00_000000 ], longestDurationTasks=[ task_1463520588324_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:48,562 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:48,562 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:48,562 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:48,562 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:48,562 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:48,562 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74] attempt: attempt_1463520588324_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:48,562 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E5F379DA932B440180AFDCD07DC9CF74
2016-05-17 21:29:48,563 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74]
2016-05-17 21:29:48,563 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:48,563 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:48,563 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588324_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:48,563 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:48,563 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520588324_0001_00_000001
2016-05-17 21:29:48,563 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:48,563 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520588324_0001_00_000001, stoppedTime=1463520588563, exitStatus=0
2016-05-17 21:29:48,565 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588324_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:48,563 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520588324_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:48,565 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520588324_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:48,565 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:48,565 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:TASK_STARTED]: vertexName=E5F379DA932B440180AFDCD07DC9CF74, taskId=task_1463520588324_0001_1_01_000000, scheduledTime=1463520588565, launchTime=1463520588565
2016-05-17 21:29:48,566 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520588324_0001_00_000001, Exiting
2016-05-17 21:29:48,566 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520588324_0001_00_000001
2016-05-17 21:29:48,566 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520588324_0001_00_000001
2016-05-17 21:29:48,566 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520588324_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:48,567 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520588324_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:48,567 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74]
2016-05-17 21:29:48,568 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588324_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:48,568 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [524C7ECDD6C545DDA1D0749C2BFFA3B0/F6D7065FFF8E45A18A5A5254A8720570] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1, VertexName: E5F379DA932B440180AFDCD07DC9CF74, VertexParallelism: 1, TaskAttemptID:attempt_1463520588324_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E98EC7CB8EA046A4BF4123D30A1D6778, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=AC4E926CEF6B42AF82C220377FAAAF72, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:48,568 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520588324_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:48,569 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:48,569 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520588324_0001_00_000002
2016-05-17 21:29:48,569 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588324_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:48,570 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520588324_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:48,570 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520588324_0001_00_000001 completed successfully
2016-05-17 21:29:48,571 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588324_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:48,571 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520588324_0001_00_000002
2016-05-17 21:29:48,572 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:48,574 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588324_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:48,574 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:48,575 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520588324_0001_00_000002, launchTime=1463520588574
2016-05-17 21:29:48,576 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520588324_0001_00_000002
2016-05-17 21:29:48,576 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520588324_0001_1_01_000000_0] to container: [container_1463520588324_0001_00_000002]
2016-05-17 21:29:48,576 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588324_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:48,577 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520588324_0001_1_01_000000_0] started. Is using containerId: [container_1463520588324_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:48,577 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E5F379DA932B440180AFDCD07DC9CF74, taskAttemptId=attempt_1463520588324_0001_1_01_000000_0, startTime=1463520588577, containerId=container_1463520588324_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520588324_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:48,577 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520588324_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:48,577 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520588324_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:48,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:48,578 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520588324_0001_00_000002 given task: attempt_1463520588324_0001_1_01_000000_0
2016-05-17 21:29:48,578 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520588324_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520588324_0001_1_01_000000_0
2016-05-17 21:29:48 Starting to run new task attempt: attempt_1463520588324_0001_1_01_000000_0
2016-05-17 21:29:48,578 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520588324_0001_1_01_000000_0
2016-05-17 21:29:48,578 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:48,578 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:48,578 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:48,579 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:48,579 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:48,580 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:48,580 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [524C7ECDD6C545DDA1D0749C2BFFA3B0/F6D7065FFF8E45A18A5A5254A8720570] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1, VertexName: E5F379DA932B440180AFDCD07DC9CF74, VertexParallelism: 1, TaskAttemptID:attempt_1463520588324_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E98EC7CB8EA046A4BF4123D30A1D6778, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=AC4E926CEF6B42AF82C220377FAAAF72, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:48,580 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:48,581 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520588324_0001_1_01_000000_0
2016-05-17 21:29:48,581 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:48,581 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:48,581 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520588324_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:48,581 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:48,581 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E98EC7CB8EA046A4BF4123D30A1D6778, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:48,582 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:48,582 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E98EC7CB8EA046A4BF4123D30A1D6778
2016-05-17 21:29:48,583 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:48,583 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:48,583 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E98EC7CB8EA046A4BF4123D30A1D6778
2016-05-17 21:29:48,583 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E98EC7CB8EA046A4BF4123D30A1D6778
2016-05-17 21:29:48,583 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AC4E926CEF6B42AF82C220377FAAAF72, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:48,584 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:48,584 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AC4E926CEF6B42AF82C220377FAAAF72
2016-05-17 21:29:48,584 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:48,584 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:48,584 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: AC4E926CEF6B42AF82C220377FAAAF72
2016-05-17 21:29:48,591 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E5F379DA932B440180AFDCD07DC9CF74, ordinal: 1
2016-05-17 21:29:48,591 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E5F379DA932B440180AFDCD07DC9CF74, mem on start (mb), free: 188, total: 363, max: 672
2016-05-17 21:29:48,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:48,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:48,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:48,593 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:48,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AC4E926CEF6B42AF82C220377FAAAF72
2016-05-17 21:29:48,595 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:48,595 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:48,595 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:48,595 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:48,595 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:48,595 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:48,595 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:48,595 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E98EC7CB8EA046A4BF4123D30A1D6778, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:48,596 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AC4E926CEF6B42AF82C220377FAAAF72, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:48,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:48,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E98EC7CB8EA046A4BF4123D30A1D6778 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:48,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:48,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:48,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E98EC7CB8EA046A4BF4123D30A1D6778
2016-05-17 21:29:48,597 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:48,597 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:48,598 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E98EC7CB8EA046A4BF4123D30A1D6778 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:48,598 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:48,598 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:48,598 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:48,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E98EC7CB8EA046A4BF4123D30A1D6778
2016-05-17 21:29:48,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:48,599 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520588324_0001_1_01_000000_0
2016-05-17 21:29:48,600 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520588324_0001_1_00_000000_0_10260, runDuration: 47324, ]
2016-05-17 21:29:48,601 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520588324_0001_1_00_000000_0_10260] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520588600, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:48,601 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E98EC7CB8EA046A4BF4123D30A1D6778
2016-05-17 21:29:48,602 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:48,602 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:48,602 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C897BB870D2A45A0A084C6DA3E1C356C
2016-05-17 21:29:48,602 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [E98EC7CB8EA046A4BF4123D30A1D6778] #44 in 2ms
2016-05-17 21:29:48,602 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_-1"], id: AC4E926CEF6B42AF82C220377FAAAF72
2016-05-17 21:29:48,603 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E98EC7CB8EA046A4BF4123D30A1D6778] #44, status:false, isInterrupted:false
2016-05-17 21:29:48,603 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_-1"]
2016-05-17 21:29:48,603 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C897BB870D2A45A0A084C6DA3E1C356C, for 1 inputs
2016-05-17 21:29:48,603 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:48,604 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:48,604 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:48,605 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:48,605 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:48,605 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:48,605 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E98EC7CB8EA046A4BF4123D30A1D6778
2016-05-17 21:29:48,605 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E5F379DA932B440180AFDCD07DC9CF74, all 1 inputs ready in: 00:00:00.002
2016-05-17 21:29:48,606 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:48,605 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:48,607 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@7b47d6ec; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@296f5f2f
2016-05-17 21:29:48,610 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:48,610 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:48,610 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520588324_0001_1_01_000000_0
2016-05-17 21:29:48,611 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520588324_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:48,611 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520588324_0001_1_01_000000_0
2016-05-17 21:29:48,611 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E5F379DA932B440180AFDCD07DC9CF74, mem on close (mb), free: 185, total: 363, max: 672
2016-05-17 21:29:48,611 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E98EC7CB8EA046A4BF4123D30A1D6778
2016-05-17 21:29:48,611 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:48,612 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:48,612 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:48,612 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:48,612 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:48,612 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:48 Completed running task attempt: attempt_1463520588324_0001_1_01_000000_0
2016-05-17 21:29:48,613 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520588324_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:48,613 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520588605, Process_Duration=6, Process_End_Time=1463520588611, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:48,614 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E5F379DA932B440180AFDCD07DC9CF74, taskAttemptId=attempt_1463520588324_0001_1_01_000000_0, startTime=1463520588577, finishTime=1463520588613, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520588605, Process_Duration=6, Process_End_Time=1463520588611, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:48,614 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:48,614 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520588324_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:48,614 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:48,614 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520588324_0001_1_01_000000_0
2016-05-17 21:29:48,614 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:48,614 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520588324_0001_00_000002
2016-05-17 21:29:48,614 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:TASK_FINISHED]: vertexName=E5F379DA932B440180AFDCD07DC9CF74, taskId=task_1463520588324_0001_1_01_000000, startTime=1463520588577, finishTime=1463520588614, timeTaken=37, status=SUCCEEDED, successfulAttemptID=attempt_1463520588324_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520588605, Process_Duration=6, Process_End_Time=1463520588611, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:48,614 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520588324_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:48,614 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:48,615 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74] : 1
2016-05-17 21:29:48,615 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74]
2016-05-17 21:29:48,615 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:48,615 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:48,615 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:VERTEX_FINISHED]: vertexName=E5F379DA932B440180AFDCD07DC9CF74, vertexId=vertex_1463520588324_0001_1_01, initRequestedTime=1463520588483, initedTime=1463520588489, startRequestedTime=1463520588496, startedTime=1463520588496, finishTime=1463520588615, timeTaken=119, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520588605, Process_Duration=6, Process_End_Time=1463520588611, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520588577, firstTasksToStart=[ task_1463520588324_0001_1_01_000000 ], lastTaskFinishTime=1463520588613, lastTasksToFinish=[ task_1463520588324_0001_1_01_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520588324_0001_1_01_000000 ], longestDurationTasks=[ task_1463520588324_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:48,616 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:48,616 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588324_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:48,616 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520588324_0001_00_000002, stoppedTime=1463520588616, exitStatus=0
2016-05-17 21:29:48,616 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588324_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:48,616 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:48,616 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520588324_0001_1
2016-05-17 21:29:48,616 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:48,616 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520588324_0001_1
2016-05-17 21:29:48,616 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520588324_0001_1_00 [E98EC7CB8EA046A4BF4123D30A1D6778]
2016-05-17 21:29:48,616 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: AC4E926CEF6B42AF82C220377FAAAF72 for vertex: vertex_1463520588324_0001_1_01 [E5F379DA932B440180AFDCD07DC9CF74]
2016-05-17 21:29:48,624 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520588324_0001_1, queueSize=0
2016-05-17 21:29:48,624 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:48,624 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588324_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520588324_0001_1, startTime=1463520588481, finishTime=1463520588623, timeTaken=142, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=345, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762314752, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041177156, Process_Duration=8, Process_End_Time=2927041177164, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:48,625 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520588324_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:48,625 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520588324_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:48,625 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520588324_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:48,625 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520588324_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:48,626 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520588324_0001_00_000002: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:48 Completed Dag: dag_1463520588324_0001_1
2016-05-17 21:29:48,627 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520588324_0001_00_000002, Exiting
2016-05-17 21:29:48,628 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520588324_0001_00_000002
2016-05-17 21:29:48,628 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520588324_0001_00_000002
2016-05-17 21:29:48,628 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520588324_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:48,628 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520588324_0001_00_000002 completed successfully
2016-05-17 21:29:48,627 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520588324_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:48,629 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520588324_0001_1_post
2016-05-17 21:29:48,629 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:48,629 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:48,629 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588324_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:48,630 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588324_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:48,633 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/ALL_UNKNOWN_-1, applicationId=application_1463520588324_0001
2016-05-17 21:29:48,634 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:48,634 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:48,634 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:48,634 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:48,634 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:48,635 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:48,635 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:48,635 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:48,635 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:48,635 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:48,635 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:48,636 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:48,636 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:48,640 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/524C7ECDD6C545DDA1D0749C2BFFA3B0/F6D7065FFF8E45A18A5A5254A8720570/.tez/application_1463520588324_0001
2016-05-17 21:29:48,640 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_-1/_temporary
2016-05-17 21:29:48,640 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.332
2016-05-17 21:29:48,702 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_-1
2016-05-17 21:29:48,721 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:48,721 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:48,748 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 21:29:48,749 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-05-17 21:29:48,750 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:48,750 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:48,750 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_'field'
2016-05-17 21:29:48,770 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:48,770 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:48,798 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 21:29:48,803 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:29:48,804 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:48,804 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:48,804 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:48,817 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:48,821 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_RESULTS
2016-05-17 21:29:48,851 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:48,851 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:48,861 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:48,862 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:48,862 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_RESULTS"]
2016-05-17 21:29:48,862 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:48,862 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:48,862 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:48,862 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-05-17 21:29:48,862 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:48,872 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:48,876 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:48,877 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:48,877 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:48,886 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1865F7D63A5B491C9FEB6D85A92FF5EB/C3903A30FD434B21AB437B083569B44C/.tez/application_1463520588876_0001 doesn't exist and is created
2016-05-17 21:29:48,900 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:48,900 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:48,901 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1865F7D63A5B491C9FEB6D85A92FF5EB/C3903A30FD434B21AB437B083569B44C/.tez/application_1463520588876_0001
2016-05-17 21:29:48,902 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520588876_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:48,913 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:48,913 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:48,914 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:48,914 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:48,914 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1865F7D63A5B491C9FEB6D85A92FF5EB/C3903A30FD434B21AB437B083569B44C/.tez/application_1463520588876_0001/localmode-log-dir
2016-05-17 21:29:48,914 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1865F7D63A5B491C9FEB6D85A92FF5EB/C3903A30FD434B21AB437B083569B44C/.tez/application_1463520588876_0001/localmode-log-dir/history.txt.appattempt_1463520588876_0001_000000, maxErrors=10
2016-05-17 21:29:48,914 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:48,914 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520588876_0001_000000, appSubmitTime=1463520588902, launchTime=1463520588902
2016-05-17 21:29:48,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:48,917 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:48,918 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:48,919 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:48,920 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:52029
2016-05-17 21:29:48,921 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:48,923 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520588876_0001_000000, startTime=1463520588923
2016-05-17 21:29:48,923 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:48,923 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:49,000 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:49,001 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:49,001 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_RESULTS, applicationId=application_1463520588876_0001, dagName=[1865F7D63A5B491C9FEB6D85A92FF5EB/C3903A30FD434B21AB437B083569B44C] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-05-17 21:29:49,013 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1865F7D63A5B491C9FEB6D85A92FF5EB/C3903A30FD434B21AB437B083569B44C] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-05-17 21:29:49,014 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520588876_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1865F7D63A5B491C9FEB6D85A92FF5EB/C3903A30FD434B21AB437B083569B44C/.tez/application_1463520588876_0001/localmode-log-dir/dag_1463520588876_0001_1.dot
2016-05-17 21:29:49,015 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1865F7D63A5B491C9FEB6D85A92FF5EB/C3903A30FD434B21AB437B083569B44C/.tez/application_1463520588876_0001/localmode-log-dir/dag_1463520588876_0001_1-tez-dag.pb.txt
2016-05-17 21:29:49,020 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520588876_0001_1
2016-05-17 21:29:49,020 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:49,021 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1865F7D63A5B491C9FEB6D85A92FF5EB/C3903A30FD434B21AB437B083569B44C] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-05-17 21:29:49 Running Dag: dag_1463520588876_0001_1
2016-05-17 21:29:49,031 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:49,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520588876_0001_1, submitTime=1463520589013
2016-05-17 21:29:49,032 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CAB23A288BED4F7BAC9A92EE2EF3C3F6
2016-05-17 21:29:49,033 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:49,033 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:49,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520588876_0001_1, initTime=1463520589032
2016-05-17 21:29:49,033 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520588876_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:49,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:DAG_STARTED]: dagID=dag_1463520588876_0001_1, startTime=1463520589033
2016-05-17 21:29:49,034 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:49,033 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:49,034 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520588876_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:49,034 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB : {F0BBDECF2C1D43E8BC270EA564B0708A={InputName=F0BBDECF2C1D43E8BC270EA564B0708A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:49,034 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F0BBDECF2C1D43E8BC270EA564B0708A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:49,034 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB]
2016-05-17 21:29:49,034 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB]
2016-05-17 21:29:49,034 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB]
2016-05-17 21:29:49,034 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB]. Starting root input initializers: 1
2016-05-17 21:29:49,035 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_RESULTS, applicationId=application_1463520588876_0001, dagName=[1865F7D63A5B491C9FEB6D85A92FF5EB/C3903A30FD434B21AB437B083569B44C] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-05-17 21:29:49,036 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:49,036 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6]
2016-05-17 21:29:49,036 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F0BBDECF2C1D43E8BC270EA564B0708A on vertex vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB]
2016-05-17 21:29:49,038 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:49,038 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F0BBDECF2C1D43E8BC270EA564B0708A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:49,038 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:49,038 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6]
2016-05-17 21:29:49,039 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6]
2016-05-17 21:29:49,039 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6]
2016-05-17 21:29:49,039 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=65499CA408594CB1A4C90CA2E58EAD2F, vertexId=vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:49,039 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=65499CA408594CB1A4C90CA2E58EAD2F, vertexId=vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6]
2016-05-17 21:29:49,038 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:49,040 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:49,040 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=65499CA408594CB1A4C90CA2E58EAD2F, vertexName=CAB23A288BED4F7BAC9A92EE2EF3C3F6, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:49,041 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=65499CA408594CB1A4C90CA2E58EAD2F, vertexId=vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6]
2016-05-17 21:29:49,042 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:49,042 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CAB23A288BED4F7BAC9A92EE2EF3C3F6, vertexId=vertex_1463520588876_0001_1_01, initRequestedTime=1463520589036, initedTime=1463520589042, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:49,042 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:49,042 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:49,043 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F0BBDECF2C1D43E8BC270EA564B0708A on vertex vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB]
2016-05-17 21:29:49,043 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:49,043 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F0BBDECF2C1D43E8BC270EA564B0708A=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:49,044 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB] parallelism set to 1
2016-05-17 21:29:49,044 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:49,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520588876_0001_1_00, eventCount=1
2016-05-17 21:29:49,044 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:49,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, vertexId=vertex_1463520588876_0001_1_00, initRequestedTime=1463520589034, initedTime=1463520589044, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:49,045 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB] with distanceFromRoot: 0
2016-05-17 21:29:49,045 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:49,045 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D03B6FEA3C214BA8BF5F5B0C4B2B6BCB
2016-05-17 21:29:49,045 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB]
2016-05-17 21:29:49,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:49,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520588876_0001_1_00, startRequestedTime=1463520589045, startedTime=1463520589045
2016-05-17 21:29:49,046 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:49,046 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:49,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:TASK_STARTED]: vertexName=D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, taskId=task_1463520588876_0001_1_00_000000, scheduledTime=1463520589046, launchTime=1463520589046
2016-05-17 21:29:49,047 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520588876_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:49,047 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520588876_0001_1_00 for vertex: vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6] numStartedSources: 1 numSources: 1
2016-05-17 21:29:49,047 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6] with distanceFromRoot: 1
2016-05-17 21:29:49,047 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB]
2016-05-17 21:29:49,047 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB in vertex: CAB23A288BED4F7BAC9A92EE2EF3C3F6
2016-05-17 21:29:49,048 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CAB23A288BED4F7BAC9A92EE2EF3C3F6 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:49,049 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:49,050 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520588876_0001_1_01, startRequestedTime=1463520589047, startedTime=1463520589047
2016-05-17 21:29:49,050 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:49,050 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1865F7D63A5B491C9FEB6D85A92FF5EB/C3903A30FD434B21AB437B083569B44C] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS, VertexName: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, VertexParallelism: 1, TaskAttemptID:attempt_1463520588876_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F0BBDECF2C1D43E8BC270EA564B0708A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CAB23A288BED4F7BAC9A92EE2EF3C3F6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:49,050 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520588876_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:49,050 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:49,051 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520588876_0001, with dag id: dag_1463520588876_0001_1
2016-05-17 21:29:49,051 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:49,051 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:49,052 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:49,052 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520588876_0001_00_000001
2016-05-17 21:29:49,052 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588876_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:49,053 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520588876_0001_00_000001
2016-05-17 21:29:49,054 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:49,054 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588876_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:49,054 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:49,055 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520588876_0001_00_000001, launchTime=1463520589054
2016-05-17 21:29:49,055 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520588876_0001_00_000001
2016-05-17 21:29:49,055 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520588876_0001_1_00_000000_0] to container: [container_1463520588876_0001_00_000001]
2016-05-17 21:29:49,056 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588876_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:49,056 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520588876_0001_1_00_000000_0] started. Is using containerId: [container_1463520588876_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:49,056 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, taskAttemptId=attempt_1463520588876_0001_1_00_000000_0, startTime=1463520589056, containerId=container_1463520588876_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520588876_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:49,056 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520588876_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:49,056 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520588876_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:49,057 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:49,057 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520588876_0001_00_000001 given task: attempt_1463520588876_0001_1_00_000000_0
2016-05-17 21:29:49,057 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520588876_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520588876_0001_1_00_000000_0
2016-05-17 21:29:49 Starting to run new task attempt: attempt_1463520588876_0001_1_00_000000_0
2016-05-17 21:29:49,057 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520588876_0001_1_00_000000_0
2016-05-17 21:29:49,057 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:49,057 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:49,058 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:49,058 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:49,058 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:49,058 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:49,058 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1865F7D63A5B491C9FEB6D85A92FF5EB/C3903A30FD434B21AB437B083569B44C] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS, VertexName: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, VertexParallelism: 1, TaskAttemptID:attempt_1463520588876_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F0BBDECF2C1D43E8BC270EA564B0708A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CAB23A288BED4F7BAC9A92EE2EF3C3F6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:49,058 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:49,058 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520588876_0001_1_00_000000_0
2016-05-17 21:29:49,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:49,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:49,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F0BBDECF2C1D43E8BC270EA564B0708A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:49,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CAB23A288BED4F7BAC9A92EE2EF3C3F6, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:49,060 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520588876_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:49,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:49,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:49,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F0BBDECF2C1D43E8BC270EA564B0708A
2016-05-17 21:29:49,061 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F0BBDECF2C1D43E8BC270EA564B0708A
2016-05-17 21:29:49,062 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:49,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:49,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CAB23A288BED4F7BAC9A92EE2EF3C3F6
2016-05-17 21:29:49,063 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:49,063 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F0BBDECF2C1D43E8BC270EA564B0708A
2016-05-17 21:29:49,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F0BBDECF2C1D43E8BC270EA564B0708A
2016-05-17 21:29:49,065 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:49,065 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CAB23A288BED4F7BAC9A92EE2EF3C3F6
2016-05-17 21:29:49,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CAB23A288BED4F7BAC9A92EE2EF3C3F6
2016-05-17 21:29:49,066 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, ordinal: 0
2016-05-17 21:29:49,066 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, mem on start (mb), free: 80, total: 363, max: 672
2016-05-17 21:29:49,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:49,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:49,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:49,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:49,068 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:49,068 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:49,068 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:49,068 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:49,069 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:49,069 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:49,069 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F0BBDECF2C1D43E8BC270EA564B0708A, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:49,069 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CAB23A288BED4F7BAC9A92EE2EF3C3F6, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:49,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:49,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F0BBDECF2C1D43E8BC270EA564B0708A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:49,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F0BBDECF2C1D43E8BC270EA564B0708A
2016-05-17 21:29:49,070 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:49,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F0BBDECF2C1D43E8BC270EA564B0708A
2016-05-17 21:29:49,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:49,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:49,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:49,071 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520588876_0001_1_00_000000_0
2016-05-17 21:29:49,071 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:49,071 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@18a89043
2016-05-17 21:29:49,072 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@18a89043
2016-05-17 21:29:49,072 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:49,072 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:49,073 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:49,074 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F0BBDECF2C1D43E8BC270EA564B0708A
2016-05-17 21:29:49,074 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 79D3CFD31F27447AB759D65B066120B2
2016-05-17 21:29:49,074 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 79D3CFD31F27447AB759D65B066120B2
2016-05-17 21:29:49,075 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:49,078 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:49,078 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@2fddde6; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@68f32892; comparator=cascading.tuple.hadoop.util.TupleComparator@33ecbf12; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:49,078 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:49,078 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:49,078 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:49,098 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:49,098 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:49,098 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:49,098 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:49,098 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:49,099 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:49,099 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:49,099 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:49,101 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520588876_0001_1_00_000000_0
2016-05-17 21:29:49,101 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, mem on close (mb), free: 180, total: 352, max: 672
2016-05-17 21:29:49,101 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:49,101 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:49,101 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:49,101 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:49,104 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:49 Completed running task attempt: attempt_1463520588876_0001_1_00_000000_0
2016-05-17 21:29:49,106 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520588876_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:49,107 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520588876_0001_1_00, eventCount=1
2016-05-17 21:29:49,107 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:49,107 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, taskAttemptId=attempt_1463520588876_0001_1_00_000000_0, startTime=1463520589056, finishTime=1463520589107, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=369623040, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520589099, Process_Duration=2, Process_End_Time=1463520589101, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:49,107 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520588876_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:49,108 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:49,108 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520588876_0001_1_00_000000_0
2016-05-17 21:29:49,108 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:49,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:TASK_FINISHED]: vertexName=D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, taskId=task_1463520588876_0001_1_00_000000, startTime=1463520589056, finishTime=1463520589108, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1463520588876_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=369623040, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520589099, Process_Duration=2, Process_End_Time=1463520589101, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:49,108 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520588876_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:49,108 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB] : 1
2016-05-17 21:29:49,108 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=369623040, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520589099, Process_Duration=2, Process_End_Time=1463520589101, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:49,108 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB]
2016-05-17 21:29:49,109 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:49,109 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:49,109 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520588876_0001_00_000001
2016-05-17 21:29:49,109 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:49,109 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:VERTEX_FINISHED]: vertexName=D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, vertexId=vertex_1463520588876_0001_1_00, initRequestedTime=1463520589034, initedTime=1463520589044, startRequestedTime=1463520589045, startedTime=1463520589045, finishTime=1463520589109, timeTaken=64, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=369623040, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520589099, Process_Duration=2, Process_End_Time=1463520589101, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520589056, firstTasksToStart=[ task_1463520588876_0001_1_00_000000 ], lastTaskFinishTime=1463520589107, lastTasksToFinish=[ task_1463520588876_0001_1_00_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520588876_0001_1_00_000000 ], longestDurationTasks=[ task_1463520588876_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:49,109 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:49,110 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588876_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:49,110 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520588876_0001_00_000001, stoppedTime=1463520589110, exitStatus=0
2016-05-17 21:29:49,110 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:49,110 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588876_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:49,111 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6] attempt: attempt_1463520588876_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:49,111 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CAB23A288BED4F7BAC9A92EE2EF3C3F6
2016-05-17 21:29:49,111 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6]
2016-05-17 21:29:49,111 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:49,112 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520588876_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:49,112 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:TASK_STARTED]: vertexName=CAB23A288BED4F7BAC9A92EE2EF3C3F6, taskId=task_1463520588876_0001_1_01_000000, scheduledTime=1463520589112, launchTime=1463520589112
2016-05-17 21:29:49,112 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:49,112 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520588876_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:49,113 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588876_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:49,113 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6]
2016-05-17 21:29:49,113 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1865F7D63A5B491C9FEB6D85A92FF5EB/C3903A30FD434B21AB437B083569B44C] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS, VertexName: CAB23A288BED4F7BAC9A92EE2EF3C3F6, VertexParallelism: 1, TaskAttemptID:attempt_1463520588876_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=65499CA408594CB1A4C90CA2E58EAD2F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:49,113 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520588876_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:49,113 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:49,114 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520588876_0001_00_000002
2016-05-17 21:29:49,114 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588876_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:49,116 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520588876_0001_00_000002
2016-05-17 21:29:49,116 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:49,116 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588876_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:49,117 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:49,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520588876_0001_00_000002, launchTime=1463520589116
2016-05-17 21:29:49,117 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520588876_0001_00_000002
2016-05-17 21:29:49,117 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520588876_0001_1_01_000000_0] to container: [container_1463520588876_0001_00_000002]
2016-05-17 21:29:49,118 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588876_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:49,118 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520588876_0001_1_01_000000_0] started. Is using containerId: [container_1463520588876_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:49,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:49,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CAB23A288BED4F7BAC9A92EE2EF3C3F6, taskAttemptId=attempt_1463520588876_0001_1_01_000000_0, startTime=1463520589118, containerId=container_1463520588876_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520588876_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:49,118 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520588876_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:49,118 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520588876_0001_00_000002 given task: attempt_1463520588876_0001_1_01_000000_0
2016-05-17 21:29:49,119 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520588876_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520588876_0001_1_01_000000_0
2016-05-17 21:29:49,119 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520588876_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:49,119 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520588876_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:49,119 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520588876_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:49 Starting to run new task attempt: attempt_1463520588876_0001_1_01_000000_0
2016-05-17 21:29:49,119 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520588876_0001_1_01_000000_0
2016-05-17 21:29:49,119 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:49,119 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520588876_0001_00_000001, Exiting
2016-05-17 21:29:49,120 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520588876_0001_00_000001
2016-05-17 21:29:49,120 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:49,120 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:49,120 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520588876_0001_00_000001
2016-05-17 21:29:49,120 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:49,120 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:49,120 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:49,120 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1865F7D63A5B491C9FEB6D85A92FF5EB/C3903A30FD434B21AB437B083569B44C] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS, VertexName: CAB23A288BED4F7BAC9A92EE2EF3C3F6, VertexParallelism: 1, TaskAttemptID:attempt_1463520588876_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=65499CA408594CB1A4C90CA2E58EAD2F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:49,120 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520588876_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:49,121 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520588876_0001_00_000001 completed successfully
2016-05-17 21:29:49,121 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:49,121 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588876_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:49,121 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520588876_0001_1_01_000000_0
2016-05-17 21:29:49,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:49,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:49,122 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520588876_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:49,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:49,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:49,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=65499CA408594CB1A4C90CA2E58EAD2F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:49,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:49,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 65499CA408594CB1A4C90CA2E58EAD2F
2016-05-17 21:29:49,123 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:49,123 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 65499CA408594CB1A4C90CA2E58EAD2F
2016-05-17 21:29:49,125 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:49,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:49,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB
2016-05-17 21:29:49,129 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:49,129 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:49,129 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB
2016-05-17 21:29:49,129 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CAB23A288BED4F7BAC9A92EE2EF3C3F6, ordinal: 1
2016-05-17 21:29:49,130 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAB23A288BED4F7BAC9A92EE2EF3C3F6, mem on start (mb), free: 174, total: 352, max: 672
2016-05-17 21:29:49,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:49,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:49,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB
2016-05-17 21:29:49,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:49,131 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:49,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 65499CA408594CB1A4C90CA2E58EAD2F
2016-05-17 21:29:49,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:49,132 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:49,132 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:49,132 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:49,132 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:49,132 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:49,132 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:49,132 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 65499CA408594CB1A4C90CA2E58EAD2F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:49,132 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D03B6FEA3C214BA8BF5F5B0C4B2B6BCB, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:49,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:49,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:49,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:49,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:49,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB
2016-05-17 21:29:49,133 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:49,134 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:49,134 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:49,134 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:49,134 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:49,135 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:49,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB
2016-05-17 21:29:49,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:49,136 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520588876_0001_1_01_000000_0
2016-05-17 21:29:49,136 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520588876_0001_1_00_000000_0_10267, runDuration: 42959, ]
2016-05-17 21:29:49,137 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520588876_0001_1_00_000000_0_10267] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520589137, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:49,137 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D03B6FEA3C214BA8BF5F5B0C4B2B6BCB
2016-05-17 21:29:49,137 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:49,138 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB] #45 in 2ms
2016-05-17 21:29:49,138 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB] #45, status:false, isInterrupted:false
2016-05-17 21:29:49,138 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:49,138 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:49,139 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 79D3CFD31F27447AB759D65B066120B2
2016-05-17 21:29:49,139 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_RESULTS"], id: 65499CA408594CB1A4C90CA2E58EAD2F
2016-05-17 21:29:49,139 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_RESULTS"]
2016-05-17 21:29:49,139 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 79D3CFD31F27447AB759D65B066120B2, for 1 inputs
2016-05-17 21:29:49,139 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:49,140 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:49,140 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:49,140 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:49,140 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:49,140 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D03B6FEA3C214BA8BF5F5B0C4B2B6BCB
2016-05-17 21:29:49,140 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CAB23A288BED4F7BAC9A92EE2EF3C3F6, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:29:49,140 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:49,141 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:49,143 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@78c52f42; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@59a1d3ff
2016-05-17 21:29:49,146 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:49,146 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:49,147 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520588876_0001_1_01_000000_0
2016-05-17 21:29:49,147 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520588876_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:49,147 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520588876_0001_1_01_000000_0
2016-05-17 21:29:49,147 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAB23A288BED4F7BAC9A92EE2EF3C3F6, mem on close (mb), free: 171, total: 352, max: 672
2016-05-17 21:29:49,148 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D03B6FEA3C214BA8BF5F5B0C4B2B6BCB
2016-05-17 21:29:49,148 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:49,149 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:49,149 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:49,149 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:49,149 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:49,149 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:49 Completed running task attempt: attempt_1463520588876_0001_1_01_000000_0
2016-05-17 21:29:49,150 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520588876_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:49,151 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CAB23A288BED4F7BAC9A92EE2EF3C3F6, taskAttemptId=attempt_1463520588876_0001_1_01_000000_0, startTime=1463520589118, finishTime=1463520589151, timeTaken=33, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=369623040, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520589140, Process_Duration=7, Process_End_Time=1463520589147, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:49,151 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520588876_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:49,151 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520588876_0001_1_01_000000_0
2016-05-17 21:29:49,152 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:49,152 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:TASK_FINISHED]: vertexName=CAB23A288BED4F7BAC9A92EE2EF3C3F6, taskId=task_1463520588876_0001_1_01_000000, startTime=1463520589118, finishTime=1463520589151, timeTaken=33, status=SUCCEEDED, successfulAttemptID=attempt_1463520588876_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=369623040, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520589140, Process_Duration=7, Process_End_Time=1463520589147, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:49,152 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:49,152 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520588876_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:49,152 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6] : 1
2016-05-17 21:29:49,152 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6]
2016-05-17 21:29:49,152 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:49,153 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:VERTEX_FINISHED]: vertexName=CAB23A288BED4F7BAC9A92EE2EF3C3F6, vertexId=vertex_1463520588876_0001_1_01, initRequestedTime=1463520589036, initedTime=1463520589042, startRequestedTime=1463520589047, startedTime=1463520589047, finishTime=1463520589152, timeTaken=105, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=369623040, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520589140, Process_Duration=7, Process_End_Time=1463520589147, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520589118, firstTasksToStart=[ task_1463520588876_0001_1_01_000000 ], lastTaskFinishTime=1463520589151, lastTasksToFinish=[ task_1463520588876_0001_1_01_000000 ], minTaskDuration=33, maxTaskDuration=33, avgTaskDuration=33.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520588876_0001_1_01_000000 ], longestDurationTasks=[ task_1463520588876_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:49,153 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:49,153 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:49,153 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=369623040, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520589140, Process_Duration=7, Process_End_Time=1463520589147, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:49,153 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588876_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:49,154 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:49,154 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:49,154 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520588876_0001_00_000002, stoppedTime=1463520589154, exitStatus=0
2016-05-17 21:29:49,154 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588876_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:49,154 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520588876_0001_00_000002
2016-05-17 21:29:49,154 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:49,154 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520588876_0001_1
2016-05-17 21:29:49,154 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520588876_0001_1
2016-05-17 21:29:49,154 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 65499CA408594CB1A4C90CA2E58EAD2F for vertex: vertex_1463520588876_0001_1_01 [CAB23A288BED4F7BAC9A92EE2EF3C3F6]
2016-05-17 21:29:49,154 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:49,155 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520588876_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:49,155 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520588876_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:49,155 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520588876_0001_00_000002, Exiting
2016-05-17 21:29:49,155 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520588876_0001_00_000002
2016-05-17 21:29:49,156 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520588876_0001_00_000002
2016-05-17 21:29:49,156 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520588876_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:49,156 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520588876_0001_00_000002 completed successfully
2016-05-17 21:29:49,162 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520588876_0001_1_00 [D03B6FEA3C214BA8BF5F5B0C4B2B6BCB]
2016-05-17 21:29:49,163 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520588876_0001_1, queueSize=0
2016-05-17 21:29:49,164 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:49,164 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520588876_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520588876_0001_1, startTime=1463520589033, finishTime=1463520589162, timeTaken=129, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=345, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=739246080, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041178239, Process_Duration=9, Process_End_Time=2927041178248, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:49,164 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520588876_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:49,164 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520588876_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:49,164 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520588876_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:49,164 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520588876_0001_00_000002
2016-05-17 21:29:49 Completed Dag: dag_1463520588876_0001_1
2016-05-17 21:29:49,165 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520588876_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:49,165 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520588876_0001_1_post
2016-05-17 21:29:49,165 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:49,166 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:49,171 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_UNKNOWN_RESULTS, applicationId=application_1463520588876_0001
2016-05-17 21:29:49,173 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:49,173 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:49,173 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:49,173 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:49,173 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:49,174 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:49,174 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:49,174 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:49,174 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:49,175 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:49,175 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:49,175 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:49,175 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:49,180 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1865F7D63A5B491C9FEB6D85A92FF5EB/C3903A30FD434B21AB437B083569B44C/.tez/application_1463520588876_0001
2016-05-17 21:29:49,181 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_RESULTS/_temporary
2016-05-17 21:29:49,181 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.320
2016-05-17 21:29:49,248 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_RESULTS
2016-05-17 21:29:49,268 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:49,268 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:49,296 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-17 21:29:49,302 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 21:29:49,302 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:49,302 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:49,302 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:49,316 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:49,320 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_ALL
2016-05-17 21:29:49,361 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:49,361 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:49,361 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_ALL"]
2016-05-17 21:29:49,361 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:49,361 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:49,361 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:49,362 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-05-17 21:29:49,362 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:49,372 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:49,376 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:49,376 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:49,376 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:49,384 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A432A0861A2E42D0B6E2284CA1B9608E/FB410F94840F4624962DC643D1B73FF0/.tez/application_1463520589376_0001 doesn't exist and is created
2016-05-17 21:29:49,399 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:49,399 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:49,399 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A432A0861A2E42D0B6E2284CA1B9608E/FB410F94840F4624962DC643D1B73FF0/.tez/application_1463520589376_0001
2016-05-17 21:29:49,401 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520589376_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:49,401 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:49,401 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:49,409 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:49,410 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:49,411 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:49,411 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:49,411 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A432A0861A2E42D0B6E2284CA1B9608E/FB410F94840F4624962DC643D1B73FF0/.tez/application_1463520589376_0001/localmode-log-dir
2016-05-17 21:29:49,411 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A432A0861A2E42D0B6E2284CA1B9608E/FB410F94840F4624962DC643D1B73FF0/.tez/application_1463520589376_0001/localmode-log-dir/history.txt.appattempt_1463520589376_0001_000000, maxErrors=10
2016-05-17 21:29:49,411 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:49,411 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520589376_0001_000000, appSubmitTime=1463520589400, launchTime=1463520589400
2016-05-17 21:29:49,414 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:49,414 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:49,415 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:49,415 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:49,416 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41611
2016-05-17 21:29:49,417 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:49,419 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520589376_0001_000000, startTime=1463520589419
2016-05-17 21:29:49,419 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:49,419 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:49,499 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:49,499 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:49,499 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/ALL_UNKNOWN_ALL, applicationId=application_1463520589376_0001, dagName=[A432A0861A2E42D0B6E2284CA1B9608E/FB410F94840F4624962DC643D1B73FF0] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-05-17 21:29:49,512 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A432A0861A2E42D0B6E2284CA1B9608E/FB410F94840F4624962DC643D1B73FF0] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-05-17 21:29:49,513 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520589376_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A432A0861A2E42D0B6E2284CA1B9608E/FB410F94840F4624962DC643D1B73FF0/.tez/application_1463520589376_0001/localmode-log-dir/dag_1463520589376_0001_1.dot
2016-05-17 21:29:49,513 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A432A0861A2E42D0B6E2284CA1B9608E/FB410F94840F4624962DC643D1B73FF0/.tez/application_1463520589376_0001/localmode-log-dir/dag_1463520589376_0001_1-tez-dag.pb.txt
2016-05-17 21:29:49,519 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520589376_0001_1
2016-05-17 21:29:49,519 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:49,519 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A432A0861A2E42D0B6E2284CA1B9608E/FB410F94840F4624962DC643D1B73FF0] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-05-17 21:29:49 Running Dag: dag_1463520589376_0001_1
2016-05-17 21:29:49,529 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520589376_0001_1, submitTime=1463520589512
2016-05-17 21:29:49,529 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:49,529 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C7E1AF4DC07A45D9ABBF752C03702955
2016-05-17 21:29:49,530 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:49,530 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520589376_0001_1, initTime=1463520589529
2016-05-17 21:29:49,530 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520589376_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:49,530 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:49,531 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:DAG_STARTED]: dagID=dag_1463520589376_0001_1, startTime=1463520589531
2016-05-17 21:29:49,531 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:49,531 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520589376_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:49,531 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F43AEE1877F14493894A59039F29F6DE : {0634986F198A40E4ADDD977DFCF1914D={InputName=0634986F198A40E4ADDD977DFCF1914D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:49,531 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0634986F198A40E4ADDD977DFCF1914D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:49,531 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE]
2016-05-17 21:29:49,531 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE]
2016-05-17 21:29:49,531 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE]
2016-05-17 21:29:49,531 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:49,531 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE]. Starting root input initializers: 1
2016-05-17 21:29:49,532 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:49,532 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955]
2016-05-17 21:29:49,532 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0634986F198A40E4ADDD977DFCF1914D on vertex vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE]
2016-05-17 21:29:49,533 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:49,533 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0634986F198A40E4ADDD977DFCF1914D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:49,534 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:49,534 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:49,535 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:49,535 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955]
2016-05-17 21:29:49,535 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955]
2016-05-17 21:29:49,535 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955]
2016-05-17 21:29:49,535 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=AA7B467593F54B7BAC03B27C97CD18FB, vertexId=vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:49,535 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=AA7B467593F54B7BAC03B27C97CD18FB, vertexId=vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955]
2016-05-17 21:29:49,535 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/ALL_UNKNOWN_ALL, applicationId=application_1463520589376_0001, dagName=[A432A0861A2E42D0B6E2284CA1B9608E/FB410F94840F4624962DC643D1B73FF0] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-05-17 21:29:49,535 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:49,536 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=AA7B467593F54B7BAC03B27C97CD18FB, vertexName=C7E1AF4DC07A45D9ABBF752C03702955, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:49,537 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=AA7B467593F54B7BAC03B27C97CD18FB, vertexId=vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955]
2016-05-17 21:29:49,537 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0634986F198A40E4ADDD977DFCF1914D on vertex vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE]
2016-05-17 21:29:49,537 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:49,538 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C7E1AF4DC07A45D9ABBF752C03702955, vertexId=vertex_1463520589376_0001_1_01, initRequestedTime=1463520589532, initedTime=1463520589537, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:49,538 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:49,538 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:49,538 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0634986F198A40E4ADDD977DFCF1914D=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:49,538 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE] parallelism set to 1
2016-05-17 21:29:49,538 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:49,538 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520589376_0001_1_00, eventCount=1
2016-05-17 21:29:49,539 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:49,539 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F43AEE1877F14493894A59039F29F6DE, vertexId=vertex_1463520589376_0001_1_00, initRequestedTime=1463520589531, initedTime=1463520589539, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:49,539 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE] with distanceFromRoot: 0
2016-05-17 21:29:49,539 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:49,539 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F43AEE1877F14493894A59039F29F6DE
2016-05-17 21:29:49,539 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE]
2016-05-17 21:29:49,540 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:49,540 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520589376_0001_1_00, startRequestedTime=1463520589539, startedTime=1463520589539
2016-05-17 21:29:49,540 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:49,541 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:49,541 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:TASK_STARTED]: vertexName=F43AEE1877F14493894A59039F29F6DE, taskId=task_1463520589376_0001_1_00_000000, scheduledTime=1463520589540, launchTime=1463520589540
2016-05-17 21:29:49,541 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520589376_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:49,541 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520589376_0001_1_00 for vertex: vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955] numStartedSources: 1 numSources: 1
2016-05-17 21:29:49,541 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955] with distanceFromRoot: 1
2016-05-17 21:29:49,541 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE]
2016-05-17 21:29:49,542 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F43AEE1877F14493894A59039F29F6DE in vertex: C7E1AF4DC07A45D9ABBF752C03702955
2016-05-17 21:29:49,542 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C7E1AF4DC07A45D9ABBF752C03702955 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:49,542 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:49,542 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520589376_0001_1_01, startRequestedTime=1463520589542, startedTime=1463520589542
2016-05-17 21:29:49,542 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:49,543 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A432A0861A2E42D0B6E2284CA1B9608E/FB410F94840F4624962DC643D1B73FF0] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL, VertexName: F43AEE1877F14493894A59039F29F6DE, VertexParallelism: 1, TaskAttemptID:attempt_1463520589376_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0634986F198A40E4ADDD977DFCF1914D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C7E1AF4DC07A45D9ABBF752C03702955, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:49,543 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520589376_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:49,543 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:49,544 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:49,544 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:49,544 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:49,545 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520589376_0001_00_000001
2016-05-17 21:29:49,545 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589376_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:49,546 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520589376_0001_00_000001
2016-05-17 21:29:49,546 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:49,547 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589376_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:49,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520589376_0001_00_000001, launchTime=1463520589547
2016-05-17 21:29:49,547 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:49,549 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520589376_0001_00_000001
2016-05-17 21:29:49,549 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520589376_0001_1_00_000000_0] to container: [container_1463520589376_0001_00_000001]
2016-05-17 21:29:49,549 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589376_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:49,550 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520589376_0001_1_00_000000_0] started. Is using containerId: [container_1463520589376_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:49,550 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F43AEE1877F14493894A59039F29F6DE, taskAttemptId=attempt_1463520589376_0001_1_00_000000_0, startTime=1463520589550, containerId=container_1463520589376_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520589376_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:49,550 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520589376_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:49,550 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520589376_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:49,550 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:49,551 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520589376_0001_00_000001 given task: attempt_1463520589376_0001_1_00_000000_0
2016-05-17 21:29:49,551 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520589376_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520589376_0001_1_00_000000_0
2016-05-17 21:29:49 Starting to run new task attempt: attempt_1463520589376_0001_1_00_000000_0
2016-05-17 21:29:49,551 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520589376_0001_1_00_000000_0
2016-05-17 21:29:49,552 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:49,552 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:49,552 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:49,552 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:49,552 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:49,552 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:49,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A432A0861A2E42D0B6E2284CA1B9608E/FB410F94840F4624962DC643D1B73FF0] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL, VertexName: F43AEE1877F14493894A59039F29F6DE, VertexParallelism: 1, TaskAttemptID:attempt_1463520589376_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0634986F198A40E4ADDD977DFCF1914D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C7E1AF4DC07A45D9ABBF752C03702955, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:49,553 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:49,553 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520589376_0001_1_00_000000_0
2016-05-17 21:29:49,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:49,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:49,554 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520589376_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:49,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:49,554 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520589376_0001, with dag id: dag_1463520589376_0001_1
2016-05-17 21:29:49,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0634986F198A40E4ADDD977DFCF1914D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:49,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:49,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0634986F198A40E4ADDD977DFCF1914D
2016-05-17 21:29:49,557 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0634986F198A40E4ADDD977DFCF1914D
2016-05-17 21:29:49,557 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:49,559 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:49,559 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0634986F198A40E4ADDD977DFCF1914D
2016-05-17 21:29:49,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0634986F198A40E4ADDD977DFCF1914D
2016-05-17 21:29:49,561 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F43AEE1877F14493894A59039F29F6DE, ordinal: 0
2016-05-17 21:29:49,561 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F43AEE1877F14493894A59039F29F6DE, mem on start (mb), free: 82, total: 352, max: 672
2016-05-17 21:29:49,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:49,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:49,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:49,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C7E1AF4DC07A45D9ABBF752C03702955, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:49,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:49,563 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C7E1AF4DC07A45D9ABBF752C03702955
2016-05-17 21:29:49,564 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:49,564 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C7E1AF4DC07A45D9ABBF752C03702955
2016-05-17 21:29:49,565 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C7E1AF4DC07A45D9ABBF752C03702955
2016-05-17 21:29:49,565 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:49,565 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:49,565 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:49,565 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:49,566 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:49,566 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:49,566 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:49,566 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0634986F198A40E4ADDD977DFCF1914D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:49,567 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C7E1AF4DC07A45D9ABBF752C03702955, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:49,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:49,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0634986F198A40E4ADDD977DFCF1914D being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:49,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0634986F198A40E4ADDD977DFCF1914D
2016-05-17 21:29:49,567 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:49,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0634986F198A40E4ADDD977DFCF1914D
2016-05-17 21:29:49,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:49,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:49,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:49,568 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520589376_0001_1_00_000000_0
2016-05-17 21:29:49,568 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:49,568 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4efc646f
2016-05-17 21:29:49,569 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4efc646f
2016-05-17 21:29:49,569 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:49,569 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:49,572 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:49,572 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0634986F198A40E4ADDD977DFCF1914D
2016-05-17 21:29:49,572 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: F7CEDC62B5FE4596BB4D968B0C9FA58E
2016-05-17 21:29:49,573 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] F7CEDC62B5FE4596BB4D968B0C9FA58E
2016-05-17 21:29:49,573 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:49,575 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:49,575 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@405219e5; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@4e222a60; comparator=cascading.tuple.hadoop.util.TupleComparator@250c2cb3; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:49,575 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:49,575 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:49,576 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:49,595 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:49,595 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:49,595 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:49,595 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:49,595 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:49,596 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:49,596 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:49,596 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F43AEE1877F14493894A59039F29F6DE, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:49,599 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520589376_0001_1_00_000000_0
2016-05-17 21:29:49,599 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F43AEE1877F14493894A59039F29F6DE, mem on close (mb), free: 190, total: 363, max: 672
2016-05-17 21:29:49,599 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:49,599 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:49,599 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:49,599 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:49,602 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:49 Completed running task attempt: attempt_1463520589376_0001_1_00_000000_0
2016-05-17 21:29:49,604 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520589376_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:49,605 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520589376_0001_1_00, eventCount=1
2016-05-17 21:29:49,605 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F43AEE1877F14493894A59039F29F6DE, taskAttemptId=attempt_1463520589376_0001_1_00_000000_0, startTime=1463520589550, finishTime=1463520589605, timeTaken=55, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520589596, Process_Duration=3, Process_End_Time=1463520589599, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:49,605 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:49,605 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520589376_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:49,605 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520589376_0001_1_00_000000_0
2016-05-17 21:29:49,605 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:49,606 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:TASK_FINISHED]: vertexName=F43AEE1877F14493894A59039F29F6DE, taskId=task_1463520589376_0001_1_00_000000, startTime=1463520589550, finishTime=1463520589605, timeTaken=55, status=SUCCEEDED, successfulAttemptID=attempt_1463520589376_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520589596, Process_Duration=3, Process_End_Time=1463520589599, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:49,606 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520589376_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:49,606 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:49,606 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE] : 1
2016-05-17 21:29:49,606 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE]
2016-05-17 21:29:49,606 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:49,606 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:VERTEX_FINISHED]: vertexName=F43AEE1877F14493894A59039F29F6DE, vertexId=vertex_1463520589376_0001_1_00, initRequestedTime=1463520589531, initedTime=1463520589539, startRequestedTime=1463520589539, startedTime=1463520589539, finishTime=1463520589606, timeTaken=67, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520589596, Process_Duration=3, Process_End_Time=1463520589599, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520589550, firstTasksToStart=[ task_1463520589376_0001_1_00_000000 ], lastTaskFinishTime=1463520589605, lastTasksToFinish=[ task_1463520589376_0001_1_00_000000 ], minTaskDuration=55, maxTaskDuration=55, avgTaskDuration=55.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520589376_0001_1_00_000000 ], longestDurationTasks=[ task_1463520589376_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:49,606 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:49,607 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589376_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:49,607 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955] attempt: attempt_1463520589376_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:49,607 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C7E1AF4DC07A45D9ABBF752C03702955
2016-05-17 21:29:49,607 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955]
2016-05-17 21:29:49,607 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:49,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520589376_0001_00_000001, stoppedTime=1463520589607, exitStatus=0
2016-05-17 21:29:49,608 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589376_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:49,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:TASK_STARTED]: vertexName=C7E1AF4DC07A45D9ABBF752C03702955, taskId=task_1463520589376_0001_1_01_000000, scheduledTime=1463520589608, launchTime=1463520589608
2016-05-17 21:29:49,608 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520589376_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:49,608 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955]
2016-05-17 21:29:49,608 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520589376_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:49,608 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A432A0861A2E42D0B6E2284CA1B9608E/FB410F94840F4624962DC643D1B73FF0] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL, VertexName: C7E1AF4DC07A45D9ABBF752C03702955, VertexParallelism: 1, TaskAttemptID:attempt_1463520589376_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F43AEE1877F14493894A59039F29F6DE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=AA7B467593F54B7BAC03B27C97CD18FB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:49,608 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520589376_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:49,608 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589376_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:49,608 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:49,608 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:49,608 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:49,609 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520589376_0001_00_000002
2016-05-17 21:29:49,609 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589376_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:49,609 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520589596, Process_Duration=3, Process_End_Time=1463520589599, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:49,611 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:49,611 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:49,611 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520589376_0001_00_000002
2016-05-17 21:29:49,611 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:49,611 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520589376_0001_00_000001
2016-05-17 21:29:49,612 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520589376_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:49,612 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520589376_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:49,613 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520589376_0001_00_000001, Exiting
2016-05-17 21:29:49,613 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520589376_0001_00_000001
2016-05-17 21:29:49,613 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520589376_0001_00_000001
2016-05-17 21:29:49,614 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589376_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:49,614 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:49,614 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520589376_0001_00_000002, launchTime=1463520589614
2016-05-17 21:29:49,615 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520589376_0001_00_000002
2016-05-17 21:29:49,616 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520589376_0001_1_01_000000_0] to container: [container_1463520589376_0001_00_000002]
2016-05-17 21:29:49,616 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589376_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:49,616 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520589376_0001_00_000002 given task: attempt_1463520589376_0001_1_01_000000_0
2016-05-17 21:29:49,616 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520589376_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520589376_0001_1_01_000000_0
2016-05-17 21:29:49,615 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520589376_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:49,616 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520589376_0001_00_000001 completed successfully
2016-05-17 21:29:49,616 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520589376_0001_1_01_000000_0] started. Is using containerId: [container_1463520589376_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:49 Starting to run new task attempt: attempt_1463520589376_0001_1_01_000000_0
2016-05-17 21:29:49,616 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520589376_0001_1_01_000000_0
2016-05-17 21:29:49,616 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:49,616 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:49,617 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:49,617 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:49,617 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:49,617 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:49,617 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A432A0861A2E42D0B6E2284CA1B9608E/FB410F94840F4624962DC643D1B73FF0] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL, VertexName: C7E1AF4DC07A45D9ABBF752C03702955, VertexParallelism: 1, TaskAttemptID:attempt_1463520589376_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F43AEE1877F14493894A59039F29F6DE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=AA7B467593F54B7BAC03B27C97CD18FB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:49,617 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:49,616 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C7E1AF4DC07A45D9ABBF752C03702955, taskAttemptId=attempt_1463520589376_0001_1_01_000000_0, startTime=1463520589616, containerId=container_1463520589376_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520589376_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:49,617 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520589376_0001_1_01_000000_0
2016-05-17 21:29:49,616 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:49,617 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520589376_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:49,618 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:49,618 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:49,618 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589376_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:49,618 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520589376_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:49,618 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520589376_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:49,619 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:49,618 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F43AEE1877F14493894A59039F29F6DE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:49,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AA7B467593F54B7BAC03B27C97CD18FB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:49,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:49,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F43AEE1877F14493894A59039F29F6DE
2016-05-17 21:29:49,621 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:49,622 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:49,622 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:49,622 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F43AEE1877F14493894A59039F29F6DE
2016-05-17 21:29:49,623 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F43AEE1877F14493894A59039F29F6DE
2016-05-17 21:29:49,623 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:49,623 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AA7B467593F54B7BAC03B27C97CD18FB
2016-05-17 21:29:49,623 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:49,623 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: AA7B467593F54B7BAC03B27C97CD18FB
2016-05-17 21:29:49,625 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C7E1AF4DC07A45D9ABBF752C03702955, ordinal: 1
2016-05-17 21:29:49,625 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C7E1AF4DC07A45D9ABBF752C03702955, mem on start (mb), free: 184, total: 363, max: 672
2016-05-17 21:29:49,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:49,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:49,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:49,632 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:49,632 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AA7B467593F54B7BAC03B27C97CD18FB
2016-05-17 21:29:49,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:49,633 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:49,633 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:49,633 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:49,633 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:49,633 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:49,633 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:49,633 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F43AEE1877F14493894A59039F29F6DE, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:49,633 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AA7B467593F54B7BAC03B27C97CD18FB, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:49,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:49,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F43AEE1877F14493894A59039F29F6DE being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:49,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:49,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:49,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F43AEE1877F14493894A59039F29F6DE
2016-05-17 21:29:49,634 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:49,634 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:49,635 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F43AEE1877F14493894A59039F29F6DE with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:49,635 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:49,635 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:49,635 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:49,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F43AEE1877F14493894A59039F29F6DE
2016-05-17 21:29:49,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:49,636 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520589376_0001_1_01_000000_0
2016-05-17 21:29:49,637 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520589376_0001_1_00_000000_0_10273, runDuration: 40836, ]
2016-05-17 21:29:49,638 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520589376_0001_1_00_000000_0_10273] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520589638, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:49,638 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F43AEE1877F14493894A59039F29F6DE
2016-05-17 21:29:49,639 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:49,639 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [F43AEE1877F14493894A59039F29F6DE] #46 in 1ms
2016-05-17 21:29:49,639 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F43AEE1877F14493894A59039F29F6DE] #46, status:false, isInterrupted:false
2016-05-17 21:29:49,639 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:49,640 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: F7CEDC62B5FE4596BB4D968B0C9FA58E
2016-05-17 21:29:49,640 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_ALL"], id: AA7B467593F54B7BAC03B27C97CD18FB
2016-05-17 21:29:49,640 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_ALL"]
2016-05-17 21:29:49,640 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] F7CEDC62B5FE4596BB4D968B0C9FA58E, for 1 inputs
2016-05-17 21:29:49,641 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:49,642 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:49,642 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:49,642 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:49,642 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:49,643 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:49,643 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F43AEE1877F14493894A59039F29F6DE
2016-05-17 21:29:49,643 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C7E1AF4DC07A45D9ABBF752C03702955, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:49,643 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:49,644 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:49,646 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@5edc9cf2; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@65c40485
2016-05-17 21:29:49,649 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:49,649 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:49,650 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520589376_0001_1_01_000000_0
2016-05-17 21:29:49,650 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520589376_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:49,650 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520589376_0001_1_01_000000_0
2016-05-17 21:29:49,651 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C7E1AF4DC07A45D9ABBF752C03702955, mem on close (mb), free: 181, total: 363, max: 672
2016-05-17 21:29:49,651 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F43AEE1877F14493894A59039F29F6DE
2016-05-17 21:29:49,651 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:49,652 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:49,652 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:49,652 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:49,652 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:49,652 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:49 Completed running task attempt: attempt_1463520589376_0001_1_01_000000_0
2016-05-17 21:29:49,653 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520589376_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:49,653 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:49,653 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C7E1AF4DC07A45D9ABBF752C03702955, taskAttemptId=attempt_1463520589376_0001_1_01_000000_0, startTime=1463520589616, finishTime=1463520589653, timeTaken=37, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=758, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520589643, Process_Duration=7, Process_End_Time=1463520589650, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:49,653 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520589376_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:49,653 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:49,653 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520589376_0001_1_01_000000_0
2016-05-17 21:29:49,654 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:49,654 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:TASK_FINISHED]: vertexName=C7E1AF4DC07A45D9ABBF752C03702955, taskId=task_1463520589376_0001_1_01_000000, startTime=1463520589616, finishTime=1463520589654, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1463520589376_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=758, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520589643, Process_Duration=7, Process_End_Time=1463520589650, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:49,654 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520589376_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:49,654 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955] : 1
2016-05-17 21:29:49,654 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955]
2016-05-17 21:29:49,655 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:49,655 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:VERTEX_FINISHED]: vertexName=C7E1AF4DC07A45D9ABBF752C03702955, vertexId=vertex_1463520589376_0001_1_01, initRequestedTime=1463520589532, initedTime=1463520589537, startRequestedTime=1463520589542, startedTime=1463520589542, finishTime=1463520589654, timeTaken=112, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=758, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520589643, Process_Duration=7, Process_End_Time=1463520589650, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520589616, firstTasksToStart=[ task_1463520589376_0001_1_01_000000 ], lastTaskFinishTime=1463520589653, lastTasksToFinish=[ task_1463520589376_0001_1_01_000000 ], minTaskDuration=37, maxTaskDuration=37, avgTaskDuration=37.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520589376_0001_1_01_000000 ], longestDurationTasks=[ task_1463520589376_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:49,655 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:49,655 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589376_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:49,656 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520589376_0001_00_000002, stoppedTime=1463520589656, exitStatus=0
2016-05-17 21:29:49,656 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:49,656 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589376_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:49,656 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:49,656 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520589376_0001_1
2016-05-17 21:29:49,656 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520589376_0001_1
2016-05-17 21:29:49,656 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: AA7B467593F54B7BAC03B27C97CD18FB for vertex: vertex_1463520589376_0001_1_01 [C7E1AF4DC07A45D9ABBF752C03702955]
2016-05-17 21:29:49,659 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=758, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520589643, Process_Duration=7, Process_End_Time=1463520589650, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:49,659 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:49,659 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:49,659 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520589376_0001_00_000002
2016-05-17 21:29:49,660 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520589376_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:49,660 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520589376_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:49,660 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520589376_0001_00_000002, Exiting
2016-05-17 21:29:49,660 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520589376_0001_00_000002
2016-05-17 21:29:49,660 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520589376_0001_00_000002
2016-05-17 21:29:49,661 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520589376_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:49,661 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520589376_0001_00_000002 completed successfully
2016-05-17 21:29:49,664 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520589376_0001_1_00 [F43AEE1877F14493894A59039F29F6DE]
2016-05-17 21:29:49,665 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520589376_0001_1, queueSize=0
2016-05-17 21:29:49,665 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:49,666 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589376_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520589376_0001_1, startTime=1463520589531, finishTime=1463520589664, timeTaken=133, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=971, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=762314752, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041179239, Process_Duration=10, Process_End_Time=2927041179249, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:49,666 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520589376_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:49,666 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520589376_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:49,666 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589376_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:49,666 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520589376_0001_00_000002
2016-05-17 21:29:49 Completed Dag: dag_1463520589376_0001_1
2016-05-17 21:29:49,666 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520589376_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:49,666 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520589376_0001_1_post
2016-05-17 21:29:49,666 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:49,666 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:49,671 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/ALL_UNKNOWN_ALL, applicationId=application_1463520589376_0001
2016-05-17 21:29:49,672 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:49,672 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:49,673 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:49,673 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:49,673 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:49,673 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:49,673 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:49,673 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:49,673 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:49,674 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:49,674 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:49,674 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:49,674 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:49,678 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/A432A0861A2E42D0B6E2284CA1B9608E/FB410F94840F4624962DC643D1B73FF0/.tez/application_1463520589376_0001
2016-05-17 21:29:49,678 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_ALL/_temporary
2016-05-17 21:29:49,678 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.317
2016-05-17 21:29:49,743 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_ALL
2016-05-17 21:29:49,762 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:49,762 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:49,788 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-17 21:29:49,793 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:29:49,793 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:49,793 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:49,793 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:49,805 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:49,809 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_REPLACE
2016-05-17 21:29:49,846 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:49,846 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:49,846 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_REPLACE"]
2016-05-17 21:29:49,846 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:49,846 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:49,847 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:49,848 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-05-17 21:29:49,848 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:49,858 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:49,862 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:49,862 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:49,862 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:49,872 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/999916319A2A44DB9856B007EDCE6B3F/D3E8130DB6764C90BEA70C0089EF173A/.tez/application_1463520589862_0001 doesn't exist and is created
2016-05-17 21:29:49,886 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:49,886 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:49,887 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/999916319A2A44DB9856B007EDCE6B3F/D3E8130DB6764C90BEA70C0089EF173A/.tez/application_1463520589862_0001
2016-05-17 21:29:49,888 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520589862_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:49,897 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:49,897 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:49,898 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:49,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:49,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/999916319A2A44DB9856B007EDCE6B3F/D3E8130DB6764C90BEA70C0089EF173A/.tez/application_1463520589862_0001/localmode-log-dir
2016-05-17 21:29:49,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/999916319A2A44DB9856B007EDCE6B3F/D3E8130DB6764C90BEA70C0089EF173A/.tez/application_1463520589862_0001/localmode-log-dir/history.txt.appattempt_1463520589862_0001_000000, maxErrors=10
2016-05-17 21:29:49,898 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:49,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520589862_0001_000000, appSubmitTime=1463520589888, launchTime=1463520589888
2016-05-17 21:29:49,902 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:49,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:49,903 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:33450
2016-05-17 21:29:49,903 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:49,905 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:49,906 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520589862_0001_000000, startTime=1463520589906
2016-05-17 21:29:49,906 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:49,907 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:49,907 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:49,986 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:49,986 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:49,987 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:49,987 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:49,987 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_REPLACE, applicationId=application_1463520589862_0001, dagName=[999916319A2A44DB9856B007EDCE6B3F/D3E8130DB6764C90BEA70C0089EF173A] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-05-17 21:29:50,001 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [999916319A2A44DB9856B007EDCE6B3F/D3E8130DB6764C90BEA70C0089EF173A] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-05-17 21:29:50,002 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520589862_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/999916319A2A44DB9856B007EDCE6B3F/D3E8130DB6764C90BEA70C0089EF173A/.tez/application_1463520589862_0001/localmode-log-dir/dag_1463520589862_0001_1.dot
2016-05-17 21:29:50,003 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/999916319A2A44DB9856B007EDCE6B3F/D3E8130DB6764C90BEA70C0089EF173A/.tez/application_1463520589862_0001/localmode-log-dir/dag_1463520589862_0001_1-tez-dag.pb.txt
2016-05-17 21:29:50,008 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520589862_0001_1
2016-05-17 21:29:50,008 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:50,008 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [999916319A2A44DB9856B007EDCE6B3F/D3E8130DB6764C90BEA70C0089EF173A] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-05-17 21:29:50 Running Dag: dag_1463520589862_0001_1
2016-05-17 21:29:50,018 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520589862_0001_1, submitTime=1463520590001
2016-05-17 21:29:50,018 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:50,018 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F08DA80AE82B49FE8AAAAED598D94325
2016-05-17 21:29:50,019 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:50,020 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520589862_0001_1, initTime=1463520590018
2016-05-17 21:29:50,020 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:50,020 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520589862_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:50,020 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:DAG_STARTED]: dagID=dag_1463520589862_0001_1, startTime=1463520590020
2016-05-17 21:29:50,020 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:50,020 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520589862_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:50,021 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E79262D0182A40C8B41A7C10FF1CB4A9 : {9D4295357075446BBC6BF9922B1C4A0A={InputName=9D4295357075446BBC6BF9922B1C4A0A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:50,021 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9D4295357075446BBC6BF9922B1C4A0A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:50,021 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9]
2016-05-17 21:29:50,021 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9]
2016-05-17 21:29:50,021 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9]
2016-05-17 21:29:50,021 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9]. Starting root input initializers: 1
2016-05-17 21:29:50,022 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:50,022 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325]
2016-05-17 21:29:50,023 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_REPLACE, applicationId=application_1463520589862_0001, dagName=[999916319A2A44DB9856B007EDCE6B3F/D3E8130DB6764C90BEA70C0089EF173A] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-05-17 21:29:50,020 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:50,022 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9D4295357075446BBC6BF9922B1C4A0A on vertex vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9]
2016-05-17 21:29:50,024 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:50,024 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325]
2016-05-17 21:29:50,024 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325]
2016-05-17 21:29:50,024 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325]
2016-05-17 21:29:50,025 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=AE45AD491A2F40379F2DA9E703C71033, vertexId=vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:50,025 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=AE45AD491A2F40379F2DA9E703C71033, vertexId=vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325]
2016-05-17 21:29:50,026 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:50,026 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9D4295357075446BBC6BF9922B1C4A0A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:50,026 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=AE45AD491A2F40379F2DA9E703C71033, vertexName=F08DA80AE82B49FE8AAAAED598D94325, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:50,026 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=AE45AD491A2F40379F2DA9E703C71033, vertexId=vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325]
2016-05-17 21:29:50,026 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:50,027 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:50,027 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:50,028 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F08DA80AE82B49FE8AAAAED598D94325, vertexId=vertex_1463520589862_0001_1_01, initRequestedTime=1463520590022, initedTime=1463520590027, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:50,028 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:50,028 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:50,029 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9D4295357075446BBC6BF9922B1C4A0A on vertex vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9]
2016-05-17 21:29:50,029 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:50,029 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9D4295357075446BBC6BF9922B1C4A0A=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:50,029 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9] parallelism set to 1
2016-05-17 21:29:50,030 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:50,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520589862_0001_1_00, eventCount=1
2016-05-17 21:29:50,030 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:50,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E79262D0182A40C8B41A7C10FF1CB4A9, vertexId=vertex_1463520589862_0001_1_00, initRequestedTime=1463520590021, initedTime=1463520590030, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:50,030 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9] with distanceFromRoot: 0
2016-05-17 21:29:50,031 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:50,031 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E79262D0182A40C8B41A7C10FF1CB4A9
2016-05-17 21:29:50,031 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9]
2016-05-17 21:29:50,031 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:50,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520589862_0001_1_00, startRequestedTime=1463520590031, startedTime=1463520590031
2016-05-17 21:29:50,032 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:50,032 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:50,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:TASK_STARTED]: vertexName=E79262D0182A40C8B41A7C10FF1CB4A9, taskId=task_1463520589862_0001_1_00_000000, scheduledTime=1463520590032, launchTime=1463520590032
2016-05-17 21:29:50,033 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520589862_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:50,033 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520589862_0001_1_00 for vertex: vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325] numStartedSources: 1 numSources: 1
2016-05-17 21:29:50,033 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325] with distanceFromRoot: 1
2016-05-17 21:29:50,033 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9]
2016-05-17 21:29:50,033 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E79262D0182A40C8B41A7C10FF1CB4A9 in vertex: F08DA80AE82B49FE8AAAAED598D94325
2016-05-17 21:29:50,033 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F08DA80AE82B49FE8AAAAED598D94325 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:50,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:50,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520589862_0001_1_01, startRequestedTime=1463520590033, startedTime=1463520590033
2016-05-17 21:29:50,034 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:50,034 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [999916319A2A44DB9856B007EDCE6B3F/D3E8130DB6764C90BEA70C0089EF173A] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE, VertexName: E79262D0182A40C8B41A7C10FF1CB4A9, VertexParallelism: 1, TaskAttemptID:attempt_1463520589862_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9D4295357075446BBC6BF9922B1C4A0A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F08DA80AE82B49FE8AAAAED598D94325, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:50,035 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520589862_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:50,035 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:50,035 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:50,036 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:50,036 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:50,036 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520589862_0001_00_000001
2016-05-17 21:29:50,036 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589862_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:50,038 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520589862_0001_00_000001
2016-05-17 21:29:50,038 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:50,038 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589862_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:50,038 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:50,038 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520589862_0001_00_000001, launchTime=1463520590038
2016-05-17 21:29:50,039 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520589862_0001_00_000001
2016-05-17 21:29:50,040 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520589862_0001_1_00_000000_0] to container: [container_1463520589862_0001_00_000001]
2016-05-17 21:29:50,040 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589862_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:50,040 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520589862_0001_00_000001 given task: attempt_1463520589862_0001_1_00_000000_0
2016-05-17 21:29:50,040 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520589862_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520589862_0001_1_00_000000_0
2016-05-17 21:29:50,040 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520589862_0001_1_00_000000_0] started. Is using containerId: [container_1463520589862_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:50 Starting to run new task attempt: attempt_1463520589862_0001_1_00_000000_0
2016-05-17 21:29:50,040 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520589862_0001_1_00_000000_0
2016-05-17 21:29:50,041 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:50,041 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:50,041 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E79262D0182A40C8B41A7C10FF1CB4A9, taskAttemptId=attempt_1463520589862_0001_1_00_000000_0, startTime=1463520590040, containerId=container_1463520589862_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520589862_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:50,041 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:50,041 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:50,041 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:50,041 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:50,041 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520589862_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:50,041 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:50,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [999916319A2A44DB9856B007EDCE6B3F/D3E8130DB6764C90BEA70C0089EF173A] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE, VertexName: E79262D0182A40C8B41A7C10FF1CB4A9, VertexParallelism: 1, TaskAttemptID:attempt_1463520589862_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9D4295357075446BBC6BF9922B1C4A0A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F08DA80AE82B49FE8AAAAED598D94325, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:50,042 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:50,042 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520589862_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:50,043 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520589862_0001, with dag id: dag_1463520589862_0001_1
2016-05-17 21:29:50,044 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520589862_0001_1_00_000000_0
2016-05-17 21:29:50,044 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:50,044 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:50,044 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520589862_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:50,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:50,046 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9D4295357075446BBC6BF9922B1C4A0A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:50,046 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:50,046 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9D4295357075446BBC6BF9922B1C4A0A
2016-05-17 21:29:50,046 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9D4295357075446BBC6BF9922B1C4A0A
2016-05-17 21:29:50,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F08DA80AE82B49FE8AAAAED598D94325, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:50,047 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:50,048 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:50,049 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9D4295357075446BBC6BF9922B1C4A0A
2016-05-17 21:29:50,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9D4295357075446BBC6BF9922B1C4A0A
2016-05-17 21:29:50,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:50,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F08DA80AE82B49FE8AAAAED598D94325
2016-05-17 21:29:50,052 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:50,053 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F08DA80AE82B49FE8AAAAED598D94325
2016-05-17 21:29:50,053 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E79262D0182A40C8B41A7C10FF1CB4A9, ordinal: 0
2016-05-17 21:29:50,053 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E79262D0182A40C8B41A7C10FF1CB4A9, mem on start (mb), free: 92, total: 363, max: 672
2016-05-17 21:29:50,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:50,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:50,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:50,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F08DA80AE82B49FE8AAAAED598D94325
2016-05-17 21:29:50,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:50,054 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:50,054 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:50,054 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:50,054 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:50,055 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:50,055 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:50,055 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9D4295357075446BBC6BF9922B1C4A0A, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:50,055 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F08DA80AE82B49FE8AAAAED598D94325, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:50,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:50,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9D4295357075446BBC6BF9922B1C4A0A being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:50,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:50,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:50,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9D4295357075446BBC6BF9922B1C4A0A
2016-05-17 21:29:50,056 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:50,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9D4295357075446BBC6BF9922B1C4A0A
2016-05-17 21:29:50,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:50,057 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520589862_0001_1_00_000000_0
2016-05-17 21:29:50,057 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:50,058 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@58e26748
2016-05-17 21:29:50,058 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@58e26748
2016-05-17 21:29:50,058 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:50,058 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:50,059 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:50,060 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 9D4295357075446BBC6BF9922B1C4A0A
2016-05-17 21:29:50,060 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C6FEBC0ADFD54F8491433BE9893ED289
2016-05-17 21:29:50,060 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C6FEBC0ADFD54F8491433BE9893ED289
2016-05-17 21:29:50,061 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:50,064 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:50,064 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@49f03bde; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@3aea91be; comparator=cascading.tuple.hadoop.util.TupleComparator@586c2d62; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:50,064 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:50,064 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:50,064 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:50,087 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:50,087 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:50,088 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:50,088 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:50,088 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:50,088 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:50,088 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:50,089 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E79262D0182A40C8B41A7C10FF1CB4A9, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:50,091 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520589862_0001_1_00_000000_0
2016-05-17 21:29:50,091 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E79262D0182A40C8B41A7C10FF1CB4A9, mem on close (mb), free: 188, total: 363, max: 672
2016-05-17 21:29:50,091 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:50,091 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:50,092 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:50,092 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:50,095 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:50 Completed running task attempt: attempt_1463520589862_0001_1_00_000000_0
2016-05-17 21:29:50,098 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520589862_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:50,098 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520589862_0001_1_00, eventCount=1
2016-05-17 21:29:50,099 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:50,099 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E79262D0182A40C8B41A7C10FF1CB4A9, taskAttemptId=attempt_1463520589862_0001_1_00_000000_0, startTime=1463520590040, finishTime=1463520590098, timeTaken=58, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380633088, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520590089, Process_Duration=2, Process_End_Time=1463520590091, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:50,099 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520589862_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:50,099 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:50,099 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520589862_0001_1_00_000000_0
2016-05-17 21:29:50,099 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:50,099 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:TASK_FINISHED]: vertexName=E79262D0182A40C8B41A7C10FF1CB4A9, taskId=task_1463520589862_0001_1_00_000000, startTime=1463520590040, finishTime=1463520590099, timeTaken=59, status=SUCCEEDED, successfulAttemptID=attempt_1463520589862_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380633088, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520590089, Process_Duration=2, Process_End_Time=1463520590091, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:50,100 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520589862_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:50,100 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9] : 1
2016-05-17 21:29:50,100 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9]
2016-05-17 21:29:50,100 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:VERTEX_FINISHED]: vertexName=E79262D0182A40C8B41A7C10FF1CB4A9, vertexId=vertex_1463520589862_0001_1_00, initRequestedTime=1463520590021, initedTime=1463520590030, startRequestedTime=1463520590031, startedTime=1463520590031, finishTime=1463520590100, timeTaken=69, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380633088, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520590089, Process_Duration=2, Process_End_Time=1463520590091, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520590040, firstTasksToStart=[ task_1463520589862_0001_1_00_000000 ], lastTaskFinishTime=1463520590098, lastTasksToFinish=[ task_1463520589862_0001_1_00_000000 ], minTaskDuration=58, maxTaskDuration=58, avgTaskDuration=58.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520589862_0001_1_00_000000 ], longestDurationTasks=[ task_1463520589862_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:50,101 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:50,101 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589862_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:50,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520589862_0001_00_000001, stoppedTime=1463520590101, exitStatus=0
2016-05-17 21:29:50,101 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589862_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:50,101 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325] attempt: attempt_1463520589862_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:50,101 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F08DA80AE82B49FE8AAAAED598D94325
2016-05-17 21:29:50,101 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325]
2016-05-17 21:29:50,101 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380633088, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520590089, Process_Duration=2, Process_End_Time=1463520590091, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:50,102 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:50,102 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:50,102 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:50,102 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:50,103 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520589862_0001_00_000001
2016-05-17 21:29:50,102 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:50,103 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520589862_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:50,103 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:TASK_STARTED]: vertexName=F08DA80AE82B49FE8AAAAED598D94325, taskId=task_1463520589862_0001_1_01_000000, scheduledTime=1463520590103, launchTime=1463520590103
2016-05-17 21:29:50,104 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520589862_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:50,104 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589862_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:50,104 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325]
2016-05-17 21:29:50,104 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [999916319A2A44DB9856B007EDCE6B3F/D3E8130DB6764C90BEA70C0089EF173A] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE, VertexName: F08DA80AE82B49FE8AAAAED598D94325, VertexParallelism: 1, TaskAttemptID:attempt_1463520589862_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E79262D0182A40C8B41A7C10FF1CB4A9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=AE45AD491A2F40379F2DA9E703C71033, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:50,104 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520589862_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:50,103 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520589862_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:50,104 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:50,103 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:50,105 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520589862_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:50,105 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520589862_0001_00_000002
2016-05-17 21:29:50,106 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589862_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:50,106 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520589862_0001_00_000001, Exiting
2016-05-17 21:29:50,106 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520589862_0001_00_000001
2016-05-17 21:29:50,106 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520589862_0001_00_000001
2016-05-17 21:29:50,107 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520589862_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:50,107 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520589862_0001_00_000002
2016-05-17 21:29:50,108 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:50,107 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520589862_0001_00_000001 completed successfully
2016-05-17 21:29:50,109 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589862_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:50,109 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589862_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:50,110 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:50,110 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520589862_0001_00_000002, launchTime=1463520590110
2016-05-17 21:29:50,110 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520589862_0001_00_000002
2016-05-17 21:29:50,111 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520589862_0001_1_01_000000_0] to container: [container_1463520589862_0001_00_000002]
2016-05-17 21:29:50,111 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589862_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:50,111 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520589862_0001_00_000002 given task: attempt_1463520589862_0001_1_01_000000_0
2016-05-17 21:29:50,111 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520589862_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520589862_0001_1_01_000000_0
2016-05-17 21:29:50,111 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520589862_0001_1_01_000000_0] started. Is using containerId: [container_1463520589862_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:50,111 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F08DA80AE82B49FE8AAAAED598D94325, taskAttemptId=attempt_1463520589862_0001_1_01_000000_0, startTime=1463520590111, containerId=container_1463520589862_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520589862_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:50,111 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:50,111 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520589862_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:50,112 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520589862_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:50 Starting to run new task attempt: attempt_1463520589862_0001_1_01_000000_0
2016-05-17 21:29:50,112 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520589862_0001_1_01_000000_0
2016-05-17 21:29:50,112 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:50,112 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:50,112 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:50,112 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:50,112 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:50,112 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:50,112 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [999916319A2A44DB9856B007EDCE6B3F/D3E8130DB6764C90BEA70C0089EF173A] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE, VertexName: F08DA80AE82B49FE8AAAAED598D94325, VertexParallelism: 1, TaskAttemptID:attempt_1463520589862_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E79262D0182A40C8B41A7C10FF1CB4A9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=AE45AD491A2F40379F2DA9E703C71033, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:50,112 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:50,113 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520589862_0001_1_01_000000_0
2016-05-17 21:29:50,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:50,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:50,113 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520589862_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:50,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:50,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E79262D0182A40C8B41A7C10FF1CB4A9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:50,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:50,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E79262D0182A40C8B41A7C10FF1CB4A9
2016-05-17 21:29:50,116 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:50,116 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:50,116 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:50,116 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E79262D0182A40C8B41A7C10FF1CB4A9
2016-05-17 21:29:50,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E79262D0182A40C8B41A7C10FF1CB4A9
2016-05-17 21:29:50,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AE45AD491A2F40379F2DA9E703C71033, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:50,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:50,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AE45AD491A2F40379F2DA9E703C71033
2016-05-17 21:29:50,118 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:50,118 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: AE45AD491A2F40379F2DA9E703C71033
2016-05-17 21:29:50,121 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F08DA80AE82B49FE8AAAAED598D94325, ordinal: 1
2016-05-17 21:29:50,121 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F08DA80AE82B49FE8AAAAED598D94325, mem on start (mb), free: 182, total: 363, max: 672
2016-05-17 21:29:50,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:50,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:50,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:50,126 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:50,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AE45AD491A2F40379F2DA9E703C71033
2016-05-17 21:29:50,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:50,127 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:50,127 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:50,127 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:50,127 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:50,127 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:50,128 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:50,128 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E79262D0182A40C8B41A7C10FF1CB4A9, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:50,128 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AE45AD491A2F40379F2DA9E703C71033, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:50,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:50,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E79262D0182A40C8B41A7C10FF1CB4A9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:50,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:50,129 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E79262D0182A40C8B41A7C10FF1CB4A9
2016-05-17 21:29:50,129 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:50,129 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:50,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:50,130 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E79262D0182A40C8B41A7C10FF1CB4A9 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:50,130 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:50,130 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:50,130 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:50,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E79262D0182A40C8B41A7C10FF1CB4A9
2016-05-17 21:29:50,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:50,131 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520589862_0001_1_01_000000_0
2016-05-17 21:29:50,133 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:50,133 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520589862_0001_1_00_000000_0_10279, runDuration: 45843, ]
2016-05-17 21:29:50,133 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C6FEBC0ADFD54F8491433BE9893ED289
2016-05-17 21:29:50,133 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_REPLACE"], id: AE45AD491A2F40379F2DA9E703C71033
2016-05-17 21:29:50,133 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_REPLACE"]
2016-05-17 21:29:50,134 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C6FEBC0ADFD54F8491433BE9893ED289, for 1 inputs
2016-05-17 21:29:50,134 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520589862_0001_1_00_000000_0_10279] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520590134, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:50,134 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E79262D0182A40C8B41A7C10FF1CB4A9
2016-05-17 21:29:50,135 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:50,135 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [E79262D0182A40C8B41A7C10FF1CB4A9] #47 in 1ms
2016-05-17 21:29:50,135 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E79262D0182A40C8B41A7C10FF1CB4A9] #47, status:false, isInterrupted:false
2016-05-17 21:29:50,136 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:50,137 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:50,137 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:50,137 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:50,137 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:50,137 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:50,137 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E79262D0182A40C8B41A7C10FF1CB4A9
2016-05-17 21:29:50,138 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:50,138 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F08DA80AE82B49FE8AAAAED598D94325, all 1 inputs ready in: 00:00:00.004
2016-05-17 21:29:50,139 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:50,140 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@55bf573b; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@4f80e78
2016-05-17 21:29:50,141 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:50,141 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:50,141 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:50,141 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:50,142 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520589862_0001_1_01_000000_0
2016-05-17 21:29:50,142 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520589862_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:50,142 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520589862_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:50,143 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F08DA80AE82B49FE8AAAAED598D94325, taskAttemptId=attempt_1463520589862_0001_1_01_000000_0, startTime=1463520590111, finishTime=1463520590143, timeTaken=32, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380633088, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520590138, Process_Duration=4, Process_End_Time=1463520590142, Read_Duration=0, Tuples_Read=1
2016-05-17 21:29:50,143 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520589862_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:50,143 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520589862_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:50,143 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:TASK_FINISHED]: vertexName=F08DA80AE82B49FE8AAAAED598D94325, taskId=task_1463520589862_0001_1_01_000000, startTime=1463520590111, finishTime=1463520590143, timeTaken=32, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:50,143 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520589862_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:50,143 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325] : 1
2016-05-17 21:29:50,143 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325] because task failed: task_1463520589862_0001_1_01_000000
2016-05-17 21:29:50,143 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:50,144 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:50,144 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325]
2016-05-17 21:29:50,143 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:50,144 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:50,144 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:50,144 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=AE45AD491A2F40379F2DA9E703C71033, vertexId=vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325]
2016-05-17 21:29:50,145 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:VERTEX_FINISHED]: vertexName=F08DA80AE82B49FE8AAAAED598D94325, vertexId=vertex_1463520589862_0001_1_01, initRequestedTime=1463520590022, initedTime=1463520590027, startRequestedTime=1463520590033, startedTime=1463520590033, finishTime=1463520590145, timeTaken=112, status=FAILED, diagnostics=Task failed, taskId=task_1463520589862_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520590111, firstTasksToStart=[ task_1463520589862_0001_1_01_000000 ], lastTaskFinishTime=1463520590143, lastTasksToFinish=[ task_1463520589862_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:50,146 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:50,146 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:50,146 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520589862_0001_00_000002, stoppedTime=1463520590146, exitStatus=0
2016-05-17 21:29:50,146 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589862_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:50,146 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520589862_0001_1_01_000000_0 failed: true container: container_1463520589862_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:50,146 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:50,147 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:50,147 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520589862_0001_1
2016-05-17 21:29:50,147 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: AE45AD491A2F40379F2DA9E703C71033 for vertex: vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325]
2016-05-17 21:29:50,147 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520589862_0001_1_00 [E79262D0182A40C8B41A7C10FF1CB4A9]
2016-05-17 21:29:50,147 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520589862_0001_1, queueSize=0
2016-05-17 21:29:50,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520589862_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520589862_0001_1, startTime=1463520590020, finishTime=1463520590147, timeTaken=127, status=FAILED, diagnostics=Vertex failed, vertexName=F08DA80AE82B49FE8AAAAED598D94325, vertexId=vertex_1463520589862_0001_1_01, diagnostics=[Task failed, taskId=task_1463520589862_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:UNKNOWN]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520589862_0001_1_01 [F08DA80AE82B49FE8AAAAED598D94325] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380633088, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520590089, Process_Duration=2, Process_End_Time=1463520590091, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:50,148 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520589862_0001_1 finished with state: FAILED
2016-05-17 21:29:50,148 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520589862_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:50,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:50,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:50,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=380633088, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520590138, Process_Duration=4, Process_End_Time=1463520590142, Read_Duration=0, Tuples_Read=1]]
2016-05-17 21:29:50,148 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F08DA80AE82B49FE8AAAAED598D94325, mem on close (mb), free: 178, total: 363, max: 672
2016-05-17 21:29:50,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=F08DA80AE82B49FE8AAAAED598D94325, index=1
2016-05-17 21:29:50,148 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520589862_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:50,149 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E79262D0182A40C8B41A7C10FF1CB4A9
2016-05-17 21:29:50,149 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:50 Completed Dag: dag_1463520589862_0001_1
2016-05-17 21:29:50,149 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520589862_0001_1, dagState=FAILED
2016-05-17 21:29:50,149 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520589862_0001_1_post
2016-05-17 21:29:50,149 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:50,149 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:50,149 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589862_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:50,150 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:50,150 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:50,150 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:50,150 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:50,150 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:50,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=F08DA80AE82B49FE8AAAAED598D94325, sourceVertex=E79262D0182A40C8B41A7C10FF1CB4A9
2016-05-17 21:29:50,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=F08DA80AE82B49FE8AAAAED598D94325, sourceVertex=AE45AD491A2F40379F2DA9E703C71033
2016-05-17 21:29:50,150 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520589862_0001_00_000002
2016-05-17 21:29:50,150 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520589862_0001_00_000002
2016-05-17 21:29:50,151 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520589862_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:50,151 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520589862_0001_00_000002 completed but with errors
2016-05-17 21:29:50,151 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520589862_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:50,151 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520589862_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:50,151 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520589862_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:50,156 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_UNKNOWN_REPLACE, applicationId=application_1463520589862_0001
2016-05-17 21:29:50,157 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:50,157 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:50,157 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:50,157 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:50,157 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:50,158 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:50,158 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:50,158 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:50,158 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:50,158 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:50,159 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:50,159 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:50,159 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:50,163 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/999916319A2A44DB9856B007EDCE6B3F/D3E8130DB6764C90BEA70C0089EF173A/.tez/application_1463520589862_0001
2016-05-17 21:29:50,163 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:50,163 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-05-17 21:29:50,164 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:50,164 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_REPLACE/_temporary
2016-05-17 21:29:50,164 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.318
2016-05-17 21:29:50,164 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_REPLACE
2016-05-17 21:29:50,185 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:50,185 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:50,211 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-17 21:29:50,219 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 21:29:50,219 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:50,219 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:50,220 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:50,232 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:50,236 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_SWAP
2016-05-17 21:29:50,274 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:50,274 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:50,275 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_SWAP"]
2016-05-17 21:29:50,275 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:50,275 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:50,276 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:50,277 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...platform/ALL_UNKNOWN_SWAP
2016-05-17 21:29:50,277 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:50,287 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:50,292 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:50,292 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:50,292 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:50,301 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/382B5A2960F04917BF4634DB0DA68677/B5DF7FB477DE441A822F27DAD0BBF5CD/.tez/application_1463520590292_0001 doesn't exist and is created
2016-05-17 21:29:50,319 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:50,319 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:50,319 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/382B5A2960F04917BF4634DB0DA68677/B5DF7FB477DE441A822F27DAD0BBF5CD/.tez/application_1463520590292_0001
2016-05-17 21:29:50,321 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520590292_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:50,330 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:50,330 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:50,331 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:50,331 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:50,331 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/382B5A2960F04917BF4634DB0DA68677/B5DF7FB477DE441A822F27DAD0BBF5CD/.tez/application_1463520590292_0001/localmode-log-dir
2016-05-17 21:29:50,331 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/382B5A2960F04917BF4634DB0DA68677/B5DF7FB477DE441A822F27DAD0BBF5CD/.tez/application_1463520590292_0001/localmode-log-dir/history.txt.appattempt_1463520590292_0001_000000, maxErrors=10
2016-05-17 21:29:50,332 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:50,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520590292_0001_000000, appSubmitTime=1463520590320, launchTime=1463520590320
2016-05-17 21:29:50,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:50,334 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:50,336 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:50,336 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:50,339 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:57825
2016-05-17 21:29:50,340 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:50,341 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520590292_0001_000000, startTime=1463520590341
2016-05-17 21:29:50,341 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:50,341 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:50,419 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:50,419 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:50,419 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_UNKNOWN_SWAP, applicationId=application_1463520590292_0001, dagName=[382B5A2960F04917BF4634DB0DA68677/B5DF7FB477DE441A822F27DAD0BBF5CD] every/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-05-17 21:29:50,433 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [382B5A2960F04917BF4634DB0DA68677/B5DF7FB477DE441A822F27DAD0BBF5CD] every/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-05-17 21:29:50,434 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520590292_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/382B5A2960F04917BF4634DB0DA68677/B5DF7FB477DE441A822F27DAD0BBF5CD/.tez/application_1463520590292_0001/localmode-log-dir/dag_1463520590292_0001_1.dot
2016-05-17 21:29:50,434 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/382B5A2960F04917BF4634DB0DA68677/B5DF7FB477DE441A822F27DAD0BBF5CD/.tez/application_1463520590292_0001/localmode-log-dir/dag_1463520590292_0001_1-tez-dag.pb.txt
2016-05-17 21:29:50,439 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520590292_0001_1
2016-05-17 21:29:50,439 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:50,439 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [382B5A2960F04917BF4634DB0DA68677/B5DF7FB477DE441A822F27DAD0BBF5CD] every/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-05-17 21:29:50 Running Dag: dag_1463520590292_0001_1
2016-05-17 21:29:50,449 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:50,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520590292_0001_1, submitTime=1463520590433
2016-05-17 21:29:50,450 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E35E2E014CB3440F977C6ABD51261A23
2016-05-17 21:29:50,450 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:50,450 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:50,450 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520590292_0001_1, initTime=1463520590449
2016-05-17 21:29:50,450 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520590292_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:50,451 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:50,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:DAG_STARTED]: dagID=dag_1463520590292_0001_1, startTime=1463520590451
2016-05-17 21:29:50,451 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:50,451 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_UNKNOWN_SWAP, applicationId=application_1463520590292_0001, dagName=[382B5A2960F04917BF4634DB0DA68677/B5DF7FB477DE441A822F27DAD0BBF5CD] every/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-05-17 21:29:50,451 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520590292_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:50,451 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D5F9655BE5CD437E88D958448A9A1ACE : {F79CD92D14F4480FA19EE047EC216819={InputName=F79CD92D14F4480FA19EE047EC216819}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:50,451 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F79CD92D14F4480FA19EE047EC216819, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:50,451 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE]
2016-05-17 21:29:50,452 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE]
2016-05-17 21:29:50,452 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE]
2016-05-17 21:29:50,452 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE]. Starting root input initializers: 1
2016-05-17 21:29:50,452 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:50,452 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23]
2016-05-17 21:29:50,452 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F79CD92D14F4480FA19EE047EC216819 on vertex vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE]
2016-05-17 21:29:50,454 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:50,454 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F79CD92D14F4480FA19EE047EC216819 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:50,454 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:50,455 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:50,455 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:50,455 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23]
2016-05-17 21:29:50,455 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23]
2016-05-17 21:29:50,456 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23]
2016-05-17 21:29:50,456 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=5A4A207FC9F54059B844C432962D9101, vertexId=vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:50,456 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=5A4A207FC9F54059B844C432962D9101, vertexId=vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23]
2016-05-17 21:29:50,457 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:50,457 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F79CD92D14F4480FA19EE047EC216819 on vertex vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE]
2016-05-17 21:29:50,458 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=5A4A207FC9F54059B844C432962D9101, vertexName=E35E2E014CB3440F977C6ABD51261A23, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:50,458 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=5A4A207FC9F54059B844C432962D9101, vertexId=vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23]
2016-05-17 21:29:50,459 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:50,459 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E35E2E014CB3440F977C6ABD51261A23, vertexId=vertex_1463520590292_0001_1_01, initRequestedTime=1463520590452, initedTime=1463520590459, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:50,460 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:50,460 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:50,460 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F79CD92D14F4480FA19EE047EC216819=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:50,460 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE] parallelism set to 1
2016-05-17 21:29:50,460 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:50,460 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520590292_0001_1_00, eventCount=1
2016-05-17 21:29:50,460 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D5F9655BE5CD437E88D958448A9A1ACE, vertexId=vertex_1463520590292_0001_1_00, initRequestedTime=1463520590451, initedTime=1463520590460, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:50,460 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE] with distanceFromRoot: 0
2016-05-17 21:29:50,460 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:50,461 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:50,461 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D5F9655BE5CD437E88D958448A9A1ACE
2016-05-17 21:29:50,461 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE]
2016-05-17 21:29:50,461 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:50,461 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520590292_0001_1_00, startRequestedTime=1463520590461, startedTime=1463520590461
2016-05-17 21:29:50,462 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:50,462 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:50,462 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:TASK_STARTED]: vertexName=D5F9655BE5CD437E88D958448A9A1ACE, taskId=task_1463520590292_0001_1_00_000000, scheduledTime=1463520590462, launchTime=1463520590462
2016-05-17 21:29:50,462 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520590292_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:50,462 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520590292_0001_1_00 for vertex: vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23] numStartedSources: 1 numSources: 1
2016-05-17 21:29:50,463 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23] with distanceFromRoot: 1
2016-05-17 21:29:50,463 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE]
2016-05-17 21:29:50,463 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D5F9655BE5CD437E88D958448A9A1ACE in vertex: E35E2E014CB3440F977C6ABD51261A23
2016-05-17 21:29:50,463 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E35E2E014CB3440F977C6ABD51261A23 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:50,464 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520590292_0001_1_01, startRequestedTime=1463520590463, startedTime=1463520590463
2016-05-17 21:29:50,464 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:50,464 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:50,464 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [382B5A2960F04917BF4634DB0DA68677/B5DF7FB477DE441A822F27DAD0BBF5CD] every/(1/1) ...platform/ALL_UNKNOWN_SWAP, VertexName: D5F9655BE5CD437E88D958448A9A1ACE, VertexParallelism: 1, TaskAttemptID:attempt_1463520590292_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F79CD92D14F4480FA19EE047EC216819, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E35E2E014CB3440F977C6ABD51261A23, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:50,464 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520590292_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:50,465 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:50,466 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:50,466 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:50,466 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:50,467 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520590292_0001_00_000001
2016-05-17 21:29:50,467 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590292_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:50,468 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520590292_0001_00_000001
2016-05-17 21:29:50,468 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:50,469 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590292_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:50,469 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520590292_0001, with dag id: dag_1463520590292_0001_1
2016-05-17 21:29:50,469 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:50,470 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520590292_0001_00_000001, launchTime=1463520590469
2016-05-17 21:29:50,470 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520590292_0001_00_000001
2016-05-17 21:29:50,471 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520590292_0001_1_00_000000_0] to container: [container_1463520590292_0001_00_000001]
2016-05-17 21:29:50,471 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590292_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:50,472 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520590292_0001_1_00_000000_0] started. Is using containerId: [container_1463520590292_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:50,472 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D5F9655BE5CD437E88D958448A9A1ACE, taskAttemptId=attempt_1463520590292_0001_1_00_000000_0, startTime=1463520590472, containerId=container_1463520590292_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520590292_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:50,472 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520590292_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:50,472 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:50,472 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520590292_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:50,472 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520590292_0001_00_000001 given task: attempt_1463520590292_0001_1_00_000000_0
2016-05-17 21:29:50,476 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520590292_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520590292_0001_1_00_000000_0
2016-05-17 21:29:50 Starting to run new task attempt: attempt_1463520590292_0001_1_00_000000_0
2016-05-17 21:29:50,476 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520590292_0001_1_00_000000_0
2016-05-17 21:29:50,476 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:50,476 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:50,476 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:50,476 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:50,476 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:50,476 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:50,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [382B5A2960F04917BF4634DB0DA68677/B5DF7FB477DE441A822F27DAD0BBF5CD] every/(1/1) ...platform/ALL_UNKNOWN_SWAP, VertexName: D5F9655BE5CD437E88D958448A9A1ACE, VertexParallelism: 1, TaskAttemptID:attempt_1463520590292_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F79CD92D14F4480FA19EE047EC216819, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E35E2E014CB3440F977C6ABD51261A23, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:50,477 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:50,477 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520590292_0001_1_00_000000_0
2016-05-17 21:29:50,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:50,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:50,477 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520590292_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:50,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F79CD92D14F4480FA19EE047EC216819, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:50,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:50,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E35E2E014CB3440F977C6ABD51261A23, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:50,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:50,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:50,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F79CD92D14F4480FA19EE047EC216819
2016-05-17 21:29:50,480 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F79CD92D14F4480FA19EE047EC216819
2016-05-17 21:29:50,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E35E2E014CB3440F977C6ABD51261A23
2016-05-17 21:29:50,482 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:50,482 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E35E2E014CB3440F977C6ABD51261A23
2016-05-17 21:29:50,482 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E35E2E014CB3440F977C6ABD51261A23
2016-05-17 21:29:50,482 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:50,482 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F79CD92D14F4480FA19EE047EC216819
2016-05-17 21:29:50,483 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:50,483 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F79CD92D14F4480FA19EE047EC216819
2016-05-17 21:29:50,487 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D5F9655BE5CD437E88D958448A9A1ACE, ordinal: 0
2016-05-17 21:29:50,487 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5F9655BE5CD437E88D958448A9A1ACE, mem on start (mb), free: 105, total: 363, max: 672
2016-05-17 21:29:50,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:50,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:50,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:50,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:50,487 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:50,487 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:50,488 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:50,488 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:50,488 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:50,488 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:50,488 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F79CD92D14F4480FA19EE047EC216819, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:50,488 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E35E2E014CB3440F977C6ABD51261A23, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:50,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:50,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F79CD92D14F4480FA19EE047EC216819 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:50,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:50,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:50,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F79CD92D14F4480FA19EE047EC216819
2016-05-17 21:29:50,489 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:50,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F79CD92D14F4480FA19EE047EC216819
2016-05-17 21:29:50,490 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:50,490 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520590292_0001_1_00_000000_0
2016-05-17 21:29:50,490 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:50,490 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3faae744
2016-05-17 21:29:50,491 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3faae744
2016-05-17 21:29:50,491 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:50,491 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:50,493 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:50,493 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F79CD92D14F4480FA19EE047EC216819
2016-05-17 21:29:50,494 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 8E756EDD5A17450AA94BD4778643748D
2016-05-17 21:29:50,494 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 8E756EDD5A17450AA94BD4778643748D
2016-05-17 21:29:50,495 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:50,498 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:50,498 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@10bf9994; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@12fbcc23; comparator=cascading.tuple.hadoop.util.TupleComparator@6187623f; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:50,498 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:50,499 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:50,499 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:50,521 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:50,522 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:50,522 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:50,522 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:50,522 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:50,523 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:50,523 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:50,523 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D5F9655BE5CD437E88D958448A9A1ACE, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:50,526 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520590292_0001_1_00_000000_0
2016-05-17 21:29:50,526 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5F9655BE5CD437E88D958448A9A1ACE, mem on close (mb), free: 188, total: 364, max: 672
2016-05-17 21:29:50,526 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:50,526 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:50,526 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:50,526 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:50,526 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:50,526 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:50,530 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:50 Completed running task attempt: attempt_1463520590292_0001_1_00_000000_0
2016-05-17 21:29:50,533 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520590292_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:50,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520590524, Process_Duration=2, Process_End_Time=1463520590526, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:50,533 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520590292_0001_1_00, eventCount=1
2016-05-17 21:29:50,534 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:50,534 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:50,534 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520590292_0001_00_000001
2016-05-17 21:29:50,536 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:50,534 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D5F9655BE5CD437E88D958448A9A1ACE, taskAttemptId=attempt_1463520590292_0001_1_00_000000_0, startTime=1463520590472, finishTime=1463520590534, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520590524, Process_Duration=2, Process_End_Time=1463520590526, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:50,537 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520590292_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:50,537 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520590292_0001_1_00_000000_0
2016-05-17 21:29:50,537 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:50,537 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:50,537 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:TASK_FINISHED]: vertexName=D5F9655BE5CD437E88D958448A9A1ACE, taskId=task_1463520590292_0001_1_00_000000, startTime=1463520590472, finishTime=1463520590537, timeTaken=65, status=SUCCEEDED, successfulAttemptID=attempt_1463520590292_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520590524, Process_Duration=2, Process_End_Time=1463520590526, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:50,537 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520590292_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:50,538 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE] : 1
2016-05-17 21:29:50,538 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE]
2016-05-17 21:29:50,539 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:VERTEX_FINISHED]: vertexName=D5F9655BE5CD437E88D958448A9A1ACE, vertexId=vertex_1463520590292_0001_1_00, initRequestedTime=1463520590451, initedTime=1463520590460, startRequestedTime=1463520590461, startedTime=1463520590461, finishTime=1463520590538, timeTaken=77, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520590524, Process_Duration=2, Process_End_Time=1463520590526, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520590472, firstTasksToStart=[ task_1463520590292_0001_1_00_000000 ], lastTaskFinishTime=1463520590534, lastTasksToFinish=[ task_1463520590292_0001_1_00_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520590292_0001_1_00_000000 ], longestDurationTasks=[ task_1463520590292_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:50,539 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:50,539 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590292_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:50,539 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520590292_0001_00_000001, stoppedTime=1463520590539, exitStatus=0
2016-05-17 21:29:50,539 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590292_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:50,539 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23] attempt: attempt_1463520590292_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:50,539 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E35E2E014CB3440F977C6ABD51261A23
2016-05-17 21:29:50,539 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23]
2016-05-17 21:29:50,539 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:50,540 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:50,540 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520590292_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:50,540 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:50,540 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:50,540 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:TASK_STARTED]: vertexName=E35E2E014CB3440F977C6ABD51261A23, taskId=task_1463520590292_0001_1_01_000000, scheduledTime=1463520590540, launchTime=1463520590540
2016-05-17 21:29:50,540 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520590292_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:50,540 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590292_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:50,541 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23]
2016-05-17 21:29:50,541 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [382B5A2960F04917BF4634DB0DA68677/B5DF7FB477DE441A822F27DAD0BBF5CD] every/(1/1) ...platform/ALL_UNKNOWN_SWAP, VertexName: E35E2E014CB3440F977C6ABD51261A23, VertexParallelism: 1, TaskAttemptID:attempt_1463520590292_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D5F9655BE5CD437E88D958448A9A1ACE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=5A4A207FC9F54059B844C432962D9101, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:50,541 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520590292_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:50,541 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:50,542 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520590292_0001_00_000002
2016-05-17 21:29:50,542 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590292_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:50,544 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520590292_0001_00_000002
2016-05-17 21:29:50,544 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:50,544 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590292_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:50,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520590292_0001_00_000002, launchTime=1463520590544
2016-05-17 21:29:50,544 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:50,545 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520590292_0001_00_000002
2016-05-17 21:29:50,546 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520590292_0001_1_01_000000_0] to container: [container_1463520590292_0001_00_000002]
2016-05-17 21:29:50,546 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590292_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:50,546 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520590292_0001_00_000002 given task: attempt_1463520590292_0001_1_01_000000_0
2016-05-17 21:29:50,546 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520590292_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520590292_0001_1_01_000000_0
2016-05-17 21:29:50 Starting to run new task attempt: attempt_1463520590292_0001_1_01_000000_0
2016-05-17 21:29:50,547 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520590292_0001_1_01_000000_0
2016-05-17 21:29:50,547 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:50,547 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:50,547 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:50,547 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:50,547 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:50,547 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:50,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [382B5A2960F04917BF4634DB0DA68677/B5DF7FB477DE441A822F27DAD0BBF5CD] every/(1/1) ...platform/ALL_UNKNOWN_SWAP, VertexName: E35E2E014CB3440F977C6ABD51261A23, VertexParallelism: 1, TaskAttemptID:attempt_1463520590292_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D5F9655BE5CD437E88D958448A9A1ACE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=5A4A207FC9F54059B844C432962D9101, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:50,547 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:50,547 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520590292_0001_1_01_000000_0] started. Is using containerId: [container_1463520590292_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:50,548 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E35E2E014CB3440F977C6ABD51261A23, taskAttemptId=attempt_1463520590292_0001_1_01_000000_0, startTime=1463520590547, containerId=container_1463520590292_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520590292_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:50,548 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:50,548 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520590292_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:50,548 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520590292_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:50,548 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520590292_0001_00_000001, Exiting
2016-05-17 21:29:50,548 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520590292_0001_00_000001
2016-05-17 21:29:50,548 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520590292_0001_00_000001
2016-05-17 21:29:50,548 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520590292_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:50,549 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520590292_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:50,548 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520590292_0001_1_01_000000_0
2016-05-17 21:29:50,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:50,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:50,550 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520590292_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:50,550 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520590292_0001_00_000001 completed successfully
2016-05-17 21:29:50,550 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590292_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:50,551 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520590292_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:50,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:50,552 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D5F9655BE5CD437E88D958448A9A1ACE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:50,552 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:50,552 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D5F9655BE5CD437E88D958448A9A1ACE
2016-05-17 21:29:50,553 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:50,553 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:50,554 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:50,555 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D5F9655BE5CD437E88D958448A9A1ACE
2016-05-17 21:29:50,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D5F9655BE5CD437E88D958448A9A1ACE
2016-05-17 21:29:50,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=5A4A207FC9F54059B844C432962D9101, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:50,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:50,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 5A4A207FC9F54059B844C432962D9101
2016-05-17 21:29:50,557 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:50,557 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E35E2E014CB3440F977C6ABD51261A23, ordinal: 1
2016-05-17 21:29:50,557 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E35E2E014CB3440F977C6ABD51261A23, mem on start (mb), free: 183, total: 364, max: 672
2016-05-17 21:29:50,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:50,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:50,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:50,557 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 5A4A207FC9F54059B844C432962D9101
2016-05-17 21:29:50,566 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:50,566 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 5A4A207FC9F54059B844C432962D9101
2016-05-17 21:29:50,567 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:50,567 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:50,567 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:50,567 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:50,567 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:50,567 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:50,567 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:50,567 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D5F9655BE5CD437E88D958448A9A1ACE, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:50,567 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 5A4A207FC9F54059B844C432962D9101, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:50,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:50,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D5F9655BE5CD437E88D958448A9A1ACE being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:50,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D5F9655BE5CD437E88D958448A9A1ACE
2016-05-17 21:29:50,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:50,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:50,568 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:50,568 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:50,568 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D5F9655BE5CD437E88D958448A9A1ACE with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:50,569 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:50,569 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:50,569 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:50,569 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D5F9655BE5CD437E88D958448A9A1ACE
2016-05-17 21:29:50,569 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:50,570 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520590292_0001_1_01_000000_0
2016-05-17 21:29:50,571 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520590292_0001_1_00_000000_0_10285, runDuration: 51286, ]
2016-05-17 21:29:50,572 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:50,572 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520590292_0001_1_00_000000_0_10285] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520590572, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:50,572 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D5F9655BE5CD437E88D958448A9A1ACE
2016-05-17 21:29:50,572 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 8E756EDD5A17450AA94BD4778643748D
2016-05-17 21:29:50,573 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_SWAP"], id: 5A4A207FC9F54059B844C432962D9101
2016-05-17 21:29:50,573 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_SWAP"]
2016-05-17 21:29:50,573 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 8E756EDD5A17450AA94BD4778643748D, for 1 inputs
2016-05-17 21:29:50,573 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:50,574 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [D5F9655BE5CD437E88D958448A9A1ACE] #48 in 3ms
2016-05-17 21:29:50,574 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D5F9655BE5CD437E88D958448A9A1ACE] #48, status:false, isInterrupted:false
2016-05-17 21:29:50,575 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:50,576 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:50,576 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:50,576 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:50,576 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:50,576 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:50,576 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E35E2E014CB3440F977C6ABD51261A23, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:50,576 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D5F9655BE5CD437E88D958448A9A1ACE
2016-05-17 21:29:50,577 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:50,578 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:50,581 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@708f56ae; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@5283f33d
2016-05-17 21:29:50,583 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:50,584 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:50,584 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520590292_0001_1_01_000000_0
2016-05-17 21:29:50,584 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520590292_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:50,586 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520590292_0001_1_01_000000_0
2016-05-17 21:29:50,586 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E35E2E014CB3440F977C6ABD51261A23, mem on close (mb), free: 179, total: 364, max: 672
2016-05-17 21:29:50,586 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D5F9655BE5CD437E88D958448A9A1ACE
2016-05-17 21:29:50,587 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:50,588 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:50,588 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:50,588 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:50,588 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:50,588 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:50 Completed running task attempt: attempt_1463520590292_0001_1_01_000000_0
2016-05-17 21:29:50,589 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520590292_0001_1_01_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:50,590 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E35E2E014CB3440F977C6ABD51261A23, taskAttemptId=attempt_1463520590292_0001_1_01_000000_0, startTime=1463520590547, finishTime=1463520590590, timeTaken=43, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520590576, Process_Duration=10, Process_End_Time=1463520590586, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:50,590 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520590292_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:50,590 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:50,590 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520590292_0001_1_01_000000_0
2016-05-17 21:29:50,590 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:50,591 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:TASK_FINISHED]: vertexName=E35E2E014CB3440F977C6ABD51261A23, taskId=task_1463520590292_0001_1_01_000000, startTime=1463520590547, finishTime=1463520590590, timeTaken=43, status=SUCCEEDED, successfulAttemptID=attempt_1463520590292_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520590576, Process_Duration=10, Process_End_Time=1463520590586, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-05-17 21:29:50,591 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520590292_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:50,591 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23] : 1
2016-05-17 21:29:50,591 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:50,591 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23]
2016-05-17 21:29:50,591 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:50,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:VERTEX_FINISHED]: vertexName=E35E2E014CB3440F977C6ABD51261A23, vertexId=vertex_1463520590292_0001_1_01, initRequestedTime=1463520590452, initedTime=1463520590459, startRequestedTime=1463520590463, startedTime=1463520590463, finishTime=1463520590591, timeTaken=128, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520590576, Process_Duration=10, Process_End_Time=1463520590586, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1463520590547, firstTasksToStart=[ task_1463520590292_0001_1_01_000000 ], lastTaskFinishTime=1463520590590, lastTasksToFinish=[ task_1463520590292_0001_1_01_000000 ], minTaskDuration=43, maxTaskDuration=43, avgTaskDuration=43.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520590292_0001_1_01_000000 ], longestDurationTasks=[ task_1463520590292_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:50,592 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:50,592 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590292_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:50,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520590292_0001_00_000002, stoppedTime=1463520590592, exitStatus=0
2016-05-17 21:29:50,592 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590292_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:50,592 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:50,593 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520590292_0001_1
2016-05-17 21:29:50,593 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1463520590292_0001_1
2016-05-17 21:29:50,593 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 5A4A207FC9F54059B844C432962D9101 for vertex: vertex_1463520590292_0001_1_01 [E35E2E014CB3440F977C6ABD51261A23]
2016-05-17 21:29:50,592 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:50,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381681664, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520590576, Process_Duration=10, Process_End_Time=1463520590586, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-05-17 21:29:50,595 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:50,595 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:50,595 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520590292_0001_00_000002
2016-05-17 21:29:50,595 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520590292_0001_00_000002 is valid, but no longer registered, and will be killed
2016-05-17 21:29:50,595 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520590292_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:50,596 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520590292_0001_00_000002, Exiting
2016-05-17 21:29:50,596 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520590292_0001_00_000002
2016-05-17 21:29:50,596 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520590292_0001_00_000002
2016-05-17 21:29:50,596 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520590292_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:50,596 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520590292_0001_00_000002 completed successfully
2016-05-17 21:29:50,601 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1463520590292_0001_1_00 [D5F9655BE5CD437E88D958448A9A1ACE]
2016-05-17 21:29:50,603 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520590292_0001_1, queueSize=0
2016-05-17 21:29:50,603 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:50,604 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590292_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520590292_0001_1, startTime=1463520590451, finishTime=1463520590602, timeTaken=151, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=322, FILE_BYTES_WRITTEN=865, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=763363328, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2927041181100, Process_Duration=12, Process_End_Time=2927041181112, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-05-17 21:29:50,604 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520590292_0001_1 finished with state: SUCCEEDED
2016-05-17 21:29:50,604 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520590292_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:50,604 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520590292_0001_00_000002
2016-05-17 21:29:50,604 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590292_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:50 Completed Dag: dag_1463520590292_0001_1
2016-05-17 21:29:50,606 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520590292_0001_1, dagState=SUCCEEDED
2016-05-17 21:29:50,606 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520590292_0001_1_post
2016-05-17 21:29:50,606 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:50,606 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:50,610 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_UNKNOWN_SWAP, applicationId=application_1463520590292_0001
2016-05-17 21:29:50,611 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:50,611 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:50,611 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:50,612 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:50,612 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:50,612 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:50,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:50,612 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:50,612 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:50,612 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:50,613 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:50,613 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:50,613 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:50,617 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/382B5A2960F04917BF4634DB0DA68677/B5DF7FB477DE441A822F27DAD0BBF5CD/.tez/application_1463520590292_0001
2016-05-17 21:29:50,617 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_SWAP/_temporary
2016-05-17 21:29:50,617 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.343
2016-05-17 21:29:50,703 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_SWAP
2016-05-17 21:29:50,722 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:50,722 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:50,750 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 21:29:50,754 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:29:50,754 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:50,754 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:50,755 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:50,767 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:50,771 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_-1
2016-05-17 21:29:50,811 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:50,811 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:50,811 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_-1"]
2016-05-17 21:29:50,812 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:50,812 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:50,812 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:50,812 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...blyplatform/ALL_VALUES_-1
2016-05-17 21:29:50,813 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:50,823 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:50,827 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:50,828 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:50,828 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:50,836 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/39003A876F4B43B9A3E01216C2F4D3E9/FE70AC05C0EE43E6BCD44FDC7E9F1DBB/.tez/application_1463520590827_0001 doesn't exist and is created
2016-05-17 21:29:50,850 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:50,851 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:50,851 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/39003A876F4B43B9A3E01216C2F4D3E9/FE70AC05C0EE43E6BCD44FDC7E9F1DBB/.tez/application_1463520590827_0001
2016-05-17 21:29:50,853 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520590827_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:50,862 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:50,863 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:50,863 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:50,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:50,864 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/39003A876F4B43B9A3E01216C2F4D3E9/FE70AC05C0EE43E6BCD44FDC7E9F1DBB/.tez/application_1463520590827_0001/localmode-log-dir
2016-05-17 21:29:50,864 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/39003A876F4B43B9A3E01216C2F4D3E9/FE70AC05C0EE43E6BCD44FDC7E9F1DBB/.tez/application_1463520590827_0001/localmode-log-dir/history.txt.appattempt_1463520590827_0001_000000, maxErrors=10
2016-05-17 21:29:50,864 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:50,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520590827_0001_000000, appSubmitTime=1463520590853, launchTime=1463520590853
2016-05-17 21:29:50,869 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:49156
2016-05-17 21:29:50,871 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:50,871 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:50,872 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:50,872 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:50,873 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:50,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520590827_0001_000000, startTime=1463520590874
2016-05-17 21:29:50,874 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:50,874 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:50,951 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:50,951 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:50,951 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_VALUES_-1, applicationId=application_1463520590827_0001, dagName=[39003A876F4B43B9A3E01216C2F4D3E9/FE70AC05C0EE43E6BCD44FDC7E9F1DBB] every/(1/1) ...blyplatform/ALL_VALUES_-1
2016-05-17 21:29:50,965 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [39003A876F4B43B9A3E01216C2F4D3E9/FE70AC05C0EE43E6BCD44FDC7E9F1DBB] every/(1/1) ...blyplatform/ALL_VALUES_-1
2016-05-17 21:29:50,966 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520590827_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/39003A876F4B43B9A3E01216C2F4D3E9/FE70AC05C0EE43E6BCD44FDC7E9F1DBB/.tez/application_1463520590827_0001/localmode-log-dir/dag_1463520590827_0001_1.dot
2016-05-17 21:29:50,967 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/39003A876F4B43B9A3E01216C2F4D3E9/FE70AC05C0EE43E6BCD44FDC7E9F1DBB/.tez/application_1463520590827_0001/localmode-log-dir/dag_1463520590827_0001_1-tez-dag.pb.txt
2016-05-17 21:29:50,972 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520590827_0001_1
2016-05-17 21:29:50,972 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:50,972 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [39003A876F4B43B9A3E01216C2F4D3E9/FE70AC05C0EE43E6BCD44FDC7E9F1DBB] every/(1/1) ...blyplatform/ALL_VALUES_-1
2016-05-17 21:29:50 Running Dag: dag_1463520590827_0001_1
2016-05-17 21:29:50,982 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520590827_0001_1, submitTime=1463520590965
2016-05-17 21:29:50,982 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:50,983 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CAF33C9BB9D6485899915E6B5A08F380
2016-05-17 21:29:50,983 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:50,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520590827_0001_1, initTime=1463520590982
2016-05-17 21:29:50,984 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520590827_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:50,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:50,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:50,984 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:DAG_STARTED]: dagID=dag_1463520590827_0001_1, startTime=1463520590984
2016-05-17 21:29:50,984 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:50,984 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520590827_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:50,984 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B17A6DFFD9794F2C8775E02E46E38952 : {BED2422A15754EE69E1731F1A569FCA9={InputName=BED2422A15754EE69E1731F1A569FCA9}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:50,984 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: BED2422A15754EE69E1731F1A569FCA9, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:50,984 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952]
2016-05-17 21:29:50,985 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952]
2016-05-17 21:29:50,985 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952]
2016-05-17 21:29:50,985 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952]. Starting root input initializers: 1
2016-05-17 21:29:50,985 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:50,985 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380]
2016-05-17 21:29:50,986 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_VALUES_-1, applicationId=application_1463520590827_0001, dagName=[39003A876F4B43B9A3E01216C2F4D3E9/FE70AC05C0EE43E6BCD44FDC7E9F1DBB] every/(1/1) ...blyplatform/ALL_VALUES_-1
2016-05-17 21:29:50,986 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: BED2422A15754EE69E1731F1A569FCA9 on vertex vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952]
2016-05-17 21:29:50,988 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:50,988 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:50,988 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380]
2016-05-17 21:29:50,988 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380]
2016-05-17 21:29:50,989 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380]
2016-05-17 21:29:50,989 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D6F45A74F12644F2B5C2123B7C34FF11, vertexId=vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:50,989 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D6F45A74F12644F2B5C2123B7C34FF11, vertexId=vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380]
2016-05-17 21:29:50,990 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input BED2422A15754EE69E1731F1A569FCA9 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:50,990 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D6F45A74F12644F2B5C2123B7C34FF11, vertexName=CAF33C9BB9D6485899915E6B5A08F380, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:50,990 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:50,990 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D6F45A74F12644F2B5C2123B7C34FF11, vertexId=vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380]
2016-05-17 21:29:50,991 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:50,991 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:50,992 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CAF33C9BB9D6485899915E6B5A08F380, vertexId=vertex_1463520590827_0001_1_01, initRequestedTime=1463520590985, initedTime=1463520590991, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:50,992 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:50,992 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:50,993 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: BED2422A15754EE69E1731F1A569FCA9 on vertex vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952]
2016-05-17 21:29:50,993 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:50,993 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {BED2422A15754EE69E1731F1A569FCA9=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:50,994 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952] parallelism set to 1
2016-05-17 21:29:50,994 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:50,995 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520590827_0001_1_00, eventCount=1
2016-05-17 21:29:50,995 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B17A6DFFD9794F2C8775E02E46E38952, vertexId=vertex_1463520590827_0001_1_00, initRequestedTime=1463520590984, initedTime=1463520590995, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:50,995 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:50,996 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952] with distanceFromRoot: 0
2016-05-17 21:29:50,996 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:50,996 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B17A6DFFD9794F2C8775E02E46E38952
2016-05-17 21:29:50,996 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952]
2016-05-17 21:29:50,997 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:50,997 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520590827_0001_1_00, startRequestedTime=1463520590996, startedTime=1463520590996
2016-05-17 21:29:50,997 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:50,998 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:50,998 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:TASK_STARTED]: vertexName=B17A6DFFD9794F2C8775E02E46E38952, taskId=task_1463520590827_0001_1_00_000000, scheduledTime=1463520590997, launchTime=1463520590997
2016-05-17 21:29:50,998 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520590827_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:50,998 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520590827_0001_1_00 for vertex: vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380] numStartedSources: 1 numSources: 1
2016-05-17 21:29:50,998 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380] with distanceFromRoot: 1
2016-05-17 21:29:50,998 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952]
2016-05-17 21:29:50,999 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B17A6DFFD9794F2C8775E02E46E38952 in vertex: CAF33C9BB9D6485899915E6B5A08F380
2016-05-17 21:29:50,999 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CAF33C9BB9D6485899915E6B5A08F380 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:50,999 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520590827_0001_1_01, startRequestedTime=1463520590999, startedTime=1463520590999
2016-05-17 21:29:50,999 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:50,999 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:50,999 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [39003A876F4B43B9A3E01216C2F4D3E9/FE70AC05C0EE43E6BCD44FDC7E9F1DBB] every/(1/1) ...blyplatform/ALL_VALUES_-1, VertexName: B17A6DFFD9794F2C8775E02E46E38952, VertexParallelism: 1, TaskAttemptID:attempt_1463520590827_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BED2422A15754EE69E1731F1A569FCA9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CAF33C9BB9D6485899915E6B5A08F380, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:50,999 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520590827_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:51,000 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:51,000 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:51,001 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:51,001 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:51,001 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520590827_0001_00_000001
2016-05-17 21:29:51,001 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590827_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:51,003 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520590827_0001_00_000001
2016-05-17 21:29:51,003 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:51,003 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590827_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:51,003 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:51,004 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520590827_0001_00_000001, launchTime=1463520591003
2016-05-17 21:29:51,004 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520590827_0001_00_000001
2016-05-17 21:29:51,004 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520590827_0001_1_00_000000_0] to container: [container_1463520590827_0001_00_000001]
2016-05-17 21:29:51,004 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590827_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:51,005 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520590827_0001_00_000001 given task: attempt_1463520590827_0001_1_00_000000_0
2016-05-17 21:29:51,005 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520590827_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520590827_0001_1_00_000000_0
2016-05-17 21:29:51,005 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520590827_0001_1_00_000000_0] started. Is using containerId: [container_1463520590827_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:51 Starting to run new task attempt: attempt_1463520590827_0001_1_00_000000_0
2016-05-17 21:29:51,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B17A6DFFD9794F2C8775E02E46E38952, taskAttemptId=attempt_1463520590827_0001_1_00_000000_0, startTime=1463520591005, containerId=container_1463520590827_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520590827_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:51,005 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520590827_0001_1_00_000000_0
2016-05-17 21:29:51,005 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:51,005 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:51,005 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:51,005 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:51,006 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:51,006 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:51,006 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [39003A876F4B43B9A3E01216C2F4D3E9/FE70AC05C0EE43E6BCD44FDC7E9F1DBB] every/(1/1) ...blyplatform/ALL_VALUES_-1, VertexName: B17A6DFFD9794F2C8775E02E46E38952, VertexParallelism: 1, TaskAttemptID:attempt_1463520590827_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BED2422A15754EE69E1731F1A569FCA9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CAF33C9BB9D6485899915E6B5A08F380, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:51,006 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:51,005 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520590827_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:51,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:51,006 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520590827_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:51,008 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520590827_0001_1_00_000000_0
2016-05-17 21:29:51,008 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:51,008 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:51,008 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520590827_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:51,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:51,009 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520590827_0001, with dag id: dag_1463520590827_0001_1
2016-05-17 21:29:51,011 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:51,011 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BED2422A15754EE69E1731F1A569FCA9, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:51,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:51,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BED2422A15754EE69E1731F1A569FCA9
2016-05-17 21:29:51,012 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: BED2422A15754EE69E1731F1A569FCA9
2016-05-17 21:29:51,014 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:51,014 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: BED2422A15754EE69E1731F1A569FCA9
2016-05-17 21:29:51,014 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BED2422A15754EE69E1731F1A569FCA9
2016-05-17 21:29:51,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CAF33C9BB9D6485899915E6B5A08F380, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:51,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:51,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CAF33C9BB9D6485899915E6B5A08F380
2016-05-17 21:29:51,015 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B17A6DFFD9794F2C8775E02E46E38952, ordinal: 0
2016-05-17 21:29:51,016 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B17A6DFFD9794F2C8775E02E46E38952, mem on start (mb), free: 89, total: 364, max: 672
2016-05-17 21:29:51,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:51,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:51,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:51,017 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:51,017 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CAF33C9BB9D6485899915E6B5A08F380
2016-05-17 21:29:51,017 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CAF33C9BB9D6485899915E6B5A08F380
2016-05-17 21:29:51,017 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:51,017 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:51,017 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:51,017 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:51,017 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:51,018 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:51,018 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:51,018 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BED2422A15754EE69E1731F1A569FCA9, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:51,018 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CAF33C9BB9D6485899915E6B5A08F380, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:51,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:51,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BED2422A15754EE69E1731F1A569FCA9 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:51,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:51,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:51,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BED2422A15754EE69E1731F1A569FCA9
2016-05-17 21:29:51,018 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:51,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BED2422A15754EE69E1731F1A569FCA9
2016-05-17 21:29:51,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:51,019 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520590827_0001_1_00_000000_0
2016-05-17 21:29:51,019 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:51,019 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2b4beaa5
2016-05-17 21:29:51,020 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2b4beaa5
2016-05-17 21:29:51,020 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:51,020 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:51,021 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:51,022 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: BED2422A15754EE69E1731F1A569FCA9
2016-05-17 21:29:51,022 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: BA83278933464EC0A9D7FBA2C1FEEEF7
2016-05-17 21:29:51,022 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] BA83278933464EC0A9D7FBA2C1FEEEF7
2016-05-17 21:29:51,023 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:51,025 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:51,025 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@1977f180; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@11901790; comparator=cascading.tuple.hadoop.util.TupleComparator@afaf7f2; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:51,025 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:51,025 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:51,025 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:51,053 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:51,053 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:51,053 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:51,053 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:51,053 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:51,054 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:51,054 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:51,055 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B17A6DFFD9794F2C8775E02E46E38952, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:51,058 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520590827_0001_1_00_000000_0
2016-05-17 21:29:51,058 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B17A6DFFD9794F2C8775E02E46E38952, mem on close (mb), free: 186, total: 363, max: 672
2016-05-17 21:29:51,058 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:51,058 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:51,058 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:51,058 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:51,061 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:51 Completed running task attempt: attempt_1463520590827_0001_1_00_000000_0
2016-05-17 21:29:51,063 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520590827_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:51,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520591055, Process_Duration=3, Process_End_Time=1463520591058, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:51,063 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520590827_0001_1_00, eventCount=1
2016-05-17 21:29:51,064 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:51,064 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:51,064 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520590827_0001_00_000001
2016-05-17 21:29:51,064 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:51,064 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:51,064 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B17A6DFFD9794F2C8775E02E46E38952, taskAttemptId=attempt_1463520590827_0001_1_00_000000_0, startTime=1463520591005, finishTime=1463520591064, timeTaken=59, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520591055, Process_Duration=3, Process_End_Time=1463520591058, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:51,064 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:51,064 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520590827_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:51,064 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:51,064 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520590827_0001_1_00_000000_0
2016-05-17 21:29:51,065 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:51,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:TASK_FINISHED]: vertexName=B17A6DFFD9794F2C8775E02E46E38952, taskId=task_1463520590827_0001_1_00_000000, startTime=1463520591005, finishTime=1463520591065, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1463520590827_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520591055, Process_Duration=3, Process_End_Time=1463520591058, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:51,065 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520590827_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:51,065 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952] : 1
2016-05-17 21:29:51,065 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952]
2016-05-17 21:29:51,065 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:51,066 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:VERTEX_FINISHED]: vertexName=B17A6DFFD9794F2C8775E02E46E38952, vertexId=vertex_1463520590827_0001_1_00, initRequestedTime=1463520590984, initedTime=1463520590995, startRequestedTime=1463520590996, startedTime=1463520590996, finishTime=1463520591065, timeTaken=69, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520591055, Process_Duration=3, Process_End_Time=1463520591058, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520591005, firstTasksToStart=[ task_1463520590827_0001_1_00_000000 ], lastTaskFinishTime=1463520591064, lastTasksToFinish=[ task_1463520590827_0001_1_00_000000 ], minTaskDuration=59, maxTaskDuration=59, avgTaskDuration=59.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520590827_0001_1_00_000000 ], longestDurationTasks=[ task_1463520590827_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:51,066 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:51,066 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590827_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:51,066 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520590827_0001_00_000001, stoppedTime=1463520591066, exitStatus=0
2016-05-17 21:29:51,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:51,066 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590827_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:51,066 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380] attempt: attempt_1463520590827_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:51,066 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CAF33C9BB9D6485899915E6B5A08F380
2016-05-17 21:29:51,066 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380]
2016-05-17 21:29:51,067 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:51,067 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520590827_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:51,067 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:51,067 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:TASK_STARTED]: vertexName=CAF33C9BB9D6485899915E6B5A08F380, taskId=task_1463520590827_0001_1_01_000000, scheduledTime=1463520591067, launchTime=1463520591067
2016-05-17 21:29:51,067 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520590827_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:51,067 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590827_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:51,068 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380]
2016-05-17 21:29:51,069 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [39003A876F4B43B9A3E01216C2F4D3E9/FE70AC05C0EE43E6BCD44FDC7E9F1DBB] every/(1/1) ...blyplatform/ALL_VALUES_-1, VertexName: CAF33C9BB9D6485899915E6B5A08F380, VertexParallelism: 1, TaskAttemptID:attempt_1463520590827_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B17A6DFFD9794F2C8775E02E46E38952, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D6F45A74F12644F2B5C2123B7C34FF11, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:51,070 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520590827_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:51,070 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:51,070 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520590827_0001_00_000002
2016-05-17 21:29:51,070 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590827_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:51,072 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520590827_0001_00_000002
2016-05-17 21:29:51,072 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:51,072 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590827_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:51,072 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:51,072 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520590827_0001_00_000002, launchTime=1463520591072
2016-05-17 21:29:51,073 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520590827_0001_00_000002
2016-05-17 21:29:51,073 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520590827_0001_1_01_000000_0] to container: [container_1463520590827_0001_00_000002]
2016-05-17 21:29:51,073 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590827_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:51,074 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520590827_0001_1_01_000000_0] started. Is using containerId: [container_1463520590827_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:51,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CAF33C9BB9D6485899915E6B5A08F380, taskAttemptId=attempt_1463520590827_0001_1_01_000000_0, startTime=1463520591074, containerId=container_1463520590827_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520590827_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:51,074 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520590827_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:51,074 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520590827_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:51,074 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520590827_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:51,074 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520590827_0001_00_000002 given task: attempt_1463520590827_0001_1_01_000000_0
2016-05-17 21:29:51,074 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:51,074 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520590827_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:51,075 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520590827_0001_00_000001, Exiting
2016-05-17 21:29:51,075 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520590827_0001_00_000001
2016-05-17 21:29:51,075 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520590827_0001_00_000001
2016-05-17 21:29:51,075 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520590827_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520590827_0001_1_01_000000_0
2016-05-17 21:29:51 Starting to run new task attempt: attempt_1463520590827_0001_1_01_000000_0
2016-05-17 21:29:51,076 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520590827_0001_1_01_000000_0
2016-05-17 21:29:51,076 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:51,076 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:51,076 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520590827_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:51,076 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520590827_0001_00_000001 completed successfully
2016-05-17 21:29:51,077 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590827_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:51,076 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:51,077 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:51,077 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:51,077 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:51,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [39003A876F4B43B9A3E01216C2F4D3E9/FE70AC05C0EE43E6BCD44FDC7E9F1DBB] every/(1/1) ...blyplatform/ALL_VALUES_-1, VertexName: CAF33C9BB9D6485899915E6B5A08F380, VertexParallelism: 1, TaskAttemptID:attempt_1463520590827_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B17A6DFFD9794F2C8775E02E46E38952, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D6F45A74F12644F2B5C2123B7C34FF11, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:51,078 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:51,078 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520590827_0001_1_01_000000_0
2016-05-17 21:29:51,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:51,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:51,079 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520590827_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:51,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B17A6DFFD9794F2C8775E02E46E38952, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:51,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:51,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B17A6DFFD9794F2C8775E02E46E38952
2016-05-17 21:29:51,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:51,082 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:51,083 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:51,083 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B17A6DFFD9794F2C8775E02E46E38952
2016-05-17 21:29:51,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B17A6DFFD9794F2C8775E02E46E38952
2016-05-17 21:29:51,083 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:51,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D6F45A74F12644F2B5C2123B7C34FF11, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:51,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:51,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D6F45A74F12644F2B5C2123B7C34FF11
2016-05-17 21:29:51,085 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:51,085 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D6F45A74F12644F2B5C2123B7C34FF11
2016-05-17 21:29:51,088 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CAF33C9BB9D6485899915E6B5A08F380, ordinal: 1
2016-05-17 21:29:51,088 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAF33C9BB9D6485899915E6B5A08F380, mem on start (mb), free: 180, total: 363, max: 672
2016-05-17 21:29:51,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:51,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:51,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:51,094 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:51,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D6F45A74F12644F2B5C2123B7C34FF11
2016-05-17 21:29:51,094 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:51,094 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:51,094 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:51,095 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:51,095 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:51,095 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:51,095 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:51,095 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B17A6DFFD9794F2C8775E02E46E38952, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:51,095 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D6F45A74F12644F2B5C2123B7C34FF11, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:51,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:51,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B17A6DFFD9794F2C8775E02E46E38952 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:51,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:51,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B17A6DFFD9794F2C8775E02E46E38952
2016-05-17 21:29:51,095 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:51,096 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:51,096 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:51,096 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B17A6DFFD9794F2C8775E02E46E38952 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:51,096 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:51,097 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:51,097 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:51,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B17A6DFFD9794F2C8775E02E46E38952
2016-05-17 21:29:51,097 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:51,098 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520590827_0001_1_01_000000_0
2016-05-17 21:29:51,098 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520590827_0001_1_00_000000_0_10291, runDuration: 46652, ]
2016-05-17 21:29:51,101 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:51,101 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: BA83278933464EC0A9D7FBA2C1FEEEF7
2016-05-17 21:29:51,101 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_-1"], id: D6F45A74F12644F2B5C2123B7C34FF11
2016-05-17 21:29:51,101 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_-1"]
2016-05-17 21:29:51,102 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] BA83278933464EC0A9D7FBA2C1FEEEF7, for 1 inputs
2016-05-17 21:29:51,103 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520590827_0001_1_00_000000_0_10291] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520591102, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:51,103 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B17A6DFFD9794F2C8775E02E46E38952
2016-05-17 21:29:51,103 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:51,103 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B17A6DFFD9794F2C8775E02E46E38952] #49 in 1ms
2016-05-17 21:29:51,104 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B17A6DFFD9794F2C8775E02E46E38952] #49, status:false, isInterrupted:false
2016-05-17 21:29:51,104 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:51,105 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:51,105 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:51,106 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:51,106 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:51,106 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:51,106 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B17A6DFFD9794F2C8775E02E46E38952
2016-05-17 21:29:51,106 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:51,106 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CAF33C9BB9D6485899915E6B5A08F380, all 1 inputs ready in: 00:00:00.004
2016-05-17 21:29:51,107 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:51,108 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@53d18ab3; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@61e918c2
2016-05-17 21:29:51,109 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:51,109 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:51,109 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:51,109 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:51,109 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520590827_0001_1_01_000000_0
2016-05-17 21:29:51,110 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520590827_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:51,110 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520590827_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:51,111 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520591106, Process_Duration=4, Process_End_Time=1463520591110, Read_Duration=1, Tuples_Read=1]]
2016-05-17 21:29:51,111 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CAF33C9BB9D6485899915E6B5A08F380, taskAttemptId=attempt_1463520590827_0001_1_01_000000_0, startTime=1463520591074, finishTime=1463520591111, timeTaken=37, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520591106, Process_Duration=4, Process_End_Time=1463520591110, Read_Duration=1, Tuples_Read=1
2016-05-17 21:29:51,111 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520590827_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:51,112 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:51,112 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520590827_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:51,112 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:TASK_FINISHED]: vertexName=CAF33C9BB9D6485899915E6B5A08F380, taskId=task_1463520590827_0001_1_01_000000, startTime=1463520591074, finishTime=1463520591112, timeTaken=38, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:51,113 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520590827_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:51,113 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380] : 1
2016-05-17 21:29:51,113 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380] because task failed: task_1463520590827_0001_1_01_000000
2016-05-17 21:29:51,113 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:51,113 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:51,113 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380]
2016-05-17 21:29:51,113 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=D6F45A74F12644F2B5C2123B7C34FF11, vertexId=vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380]
2016-05-17 21:29:51,111 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CAF33C9BB9D6485899915E6B5A08F380, mem on close (mb), free: 177, total: 363, max: 672
2016-05-17 21:29:51,113 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:51,113 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:51,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=CAF33C9BB9D6485899915E6B5A08F380, index=1
2016-05-17 21:29:51,114 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B17A6DFFD9794F2C8775E02E46E38952
2016-05-17 21:29:51,114 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:51,114 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:VERTEX_FINISHED]: vertexName=CAF33C9BB9D6485899915E6B5A08F380, vertexId=vertex_1463520590827_0001_1_01, initRequestedTime=1463520590985, initedTime=1463520590991, startRequestedTime=1463520590999, startedTime=1463520590999, finishTime=1463520591114, timeTaken=115, status=FAILED, diagnostics=Task failed, taskId=task_1463520590827_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520591074, firstTasksToStart=[ task_1463520590827_0001_1_01_000000 ], lastTaskFinishTime=1463520591111, lastTasksToFinish=[ task_1463520590827_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:51,115 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:51,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520590827_0001_00_000002, stoppedTime=1463520591115, exitStatus=0
2016-05-17 21:29:51,115 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590827_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:51,115 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520590827_0001_1_01_000000_0 failed: true container: container_1463520590827_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:51,115 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:51,115 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:51,115 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520590827_0001_1
2016-05-17 21:29:51,115 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:51,116 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:51,116 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520590827_0001_1_00 [B17A6DFFD9794F2C8775E02E46E38952]
2016-05-17 21:29:51,116 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: D6F45A74F12644F2B5C2123B7C34FF11 for vertex: vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380]
2016-05-17 21:29:51,117 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520590827_0001_1, queueSize=0
2016-05-17 21:29:51,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:51,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520590827_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520590827_0001_1, startTime=1463520590984, finishTime=1463520591116, timeTaken=132, status=FAILED, diagnostics=Vertex failed, vertexName=CAF33C9BB9D6485899915E6B5A08F380, vertexId=vertex_1463520590827_0001_1_01, diagnostics=[Task failed, taskId=task_1463520590827_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520590827_0001_1_01 [CAF33C9BB9D6485899915E6B5A08F380] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=733, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=13, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=381157376, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520591055, Process_Duration=3, Process_End_Time=1463520591058, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:51,118 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520590827_0001_1 finished with state: FAILED
2016-05-17 21:29:51,118 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520590827_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:51,119 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520590827_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:51 Completed Dag: dag_1463520590827_0001_1
2016-05-17 21:29:51,119 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520590827_0001_1, dagState=FAILED
2016-05-17 21:29:51,119 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520590827_0001_1_post
2016-05-17 21:29:51,119 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:51,119 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:51,120 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590827_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:51,121 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:51,121 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:51,121 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:51,122 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:51,122 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:51,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=CAF33C9BB9D6485899915E6B5A08F380, sourceVertex=B17A6DFFD9794F2C8775E02E46E38952
2016-05-17 21:29:51,123 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/ALL_VALUES_-1, applicationId=application_1463520590827_0001
2016-05-17 21:29:51,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=CAF33C9BB9D6485899915E6B5A08F380, sourceVertex=D6F45A74F12644F2B5C2123B7C34FF11
2016-05-17 21:29:51,123 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520590827_0001_00_000002
2016-05-17 21:29:51,123 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520590827_0001_00_000002
2016-05-17 21:29:51,124 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520590827_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:51,124 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520590827_0001_00_000002 completed but with errors
2016-05-17 21:29:51,124 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520590827_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:51,124 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520590827_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:51,124 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520590827_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:51,125 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:51,125 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:51,125 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:51,125 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:51,125 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:51,125 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:51,126 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:51,126 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:51,126 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:51,126 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:51,126 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:51,126 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:51,126 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:51,130 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/39003A876F4B43B9A3E01216C2F4D3E9/FE70AC05C0EE43E6BCD44FDC7E9F1DBB/.tez/application_1463520590827_0001
2016-05-17 21:29:51,130 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:51,131 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...blyplatform/ALL_VALUES_-1
2016-05-17 21:29:51,131 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:51,131 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_-1/_temporary
2016-05-17 21:29:51,132 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.320
2016-05-17 21:29:51,132 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_-1
2016-05-17 21:29:51,153 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:51,153 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:51,176 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-05-17 21:29:51,177 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 21:29:51,178 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:51,178 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:51,178 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_'field'
2016-05-17 21:29:51,196 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:51,196 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:51,222 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-17 21:29:51,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:29:51,227 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:51,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:51,228 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:51,240 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:51,244 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_RESULTS
2016-05-17 21:29:51,286 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:51,286 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:51,286 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_RESULTS"]
2016-05-17 21:29:51,286 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:51,286 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:51,286 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:51,287 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...atform/ALL_VALUES_RESULTS
2016-05-17 21:29:51,287 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:51,298 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:51,303 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:51,304 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:51,304 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:51,313 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1FE18A7514BE48F5BEA72795AD8B281D/CE3D68EE69F542D6A7A68CBC22D1E3DF/.tez/application_1463520591303_0001 doesn't exist and is created
2016-05-17 21:29:51,328 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:51,328 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:51,328 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1FE18A7514BE48F5BEA72795AD8B281D/CE3D68EE69F542D6A7A68CBC22D1E3DF/.tez/application_1463520591303_0001
2016-05-17 21:29:51,330 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520591303_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:51,338 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:51,339 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:51,340 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:51,340 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:51,340 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1FE18A7514BE48F5BEA72795AD8B281D/CE3D68EE69F542D6A7A68CBC22D1E3DF/.tez/application_1463520591303_0001/localmode-log-dir
2016-05-17 21:29:51,340 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1FE18A7514BE48F5BEA72795AD8B281D/CE3D68EE69F542D6A7A68CBC22D1E3DF/.tez/application_1463520591303_0001/localmode-log-dir/history.txt.appattempt_1463520591303_0001_000000, maxErrors=10
2016-05-17 21:29:51,340 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:51,340 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520591303_0001_000000, appSubmitTime=1463520591329, launchTime=1463520591329
2016-05-17 21:29:51,344 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:51,345 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:51,345 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:51,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:51,346 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:51983
2016-05-17 21:29:51,347 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:51,349 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520591303_0001_000000, startTime=1463520591348
2016-05-17 21:29:51,349 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:51,349 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:51,428 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:51,428 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:51,428 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_RESULTS, applicationId=application_1463520591303_0001, dagName=[1FE18A7514BE48F5BEA72795AD8B281D/CE3D68EE69F542D6A7A68CBC22D1E3DF] every/(1/1) ...atform/ALL_VALUES_RESULTS
2016-05-17 21:29:51,442 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1FE18A7514BE48F5BEA72795AD8B281D/CE3D68EE69F542D6A7A68CBC22D1E3DF] every/(1/1) ...atform/ALL_VALUES_RESULTS
2016-05-17 21:29:51,444 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520591303_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1FE18A7514BE48F5BEA72795AD8B281D/CE3D68EE69F542D6A7A68CBC22D1E3DF/.tez/application_1463520591303_0001/localmode-log-dir/dag_1463520591303_0001_1.dot
2016-05-17 21:29:51,444 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1FE18A7514BE48F5BEA72795AD8B281D/CE3D68EE69F542D6A7A68CBC22D1E3DF/.tez/application_1463520591303_0001/localmode-log-dir/dag_1463520591303_0001_1-tez-dag.pb.txt
2016-05-17 21:29:51,449 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520591303_0001_1
2016-05-17 21:29:51,449 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:51,449 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1FE18A7514BE48F5BEA72795AD8B281D/CE3D68EE69F542D6A7A68CBC22D1E3DF] every/(1/1) ...atform/ALL_VALUES_RESULTS
2016-05-17 21:29:51 Running Dag: dag_1463520591303_0001_1
2016-05-17 21:29:51,460 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520591303_0001_1, submitTime=1463520591442
2016-05-17 21:29:51,460 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:51,460 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F36BAD8E92DF441CBCB6172E852450BC
2016-05-17 21:29:51,461 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:51,461 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520591303_0001_1, initTime=1463520591460
2016-05-17 21:29:51,461 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:51,461 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520591303_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:51,462 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:DAG_STARTED]: dagID=dag_1463520591303_0001_1, startTime=1463520591462
2016-05-17 21:29:51,462 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:51,462 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520591303_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:51,462 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C492EE4B7C1047E0818C6C2A356AA5E6 : {D9B7D45670364138B7FCE1BE523A3284={InputName=D9B7D45670364138B7FCE1BE523A3284}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:51,462 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: D9B7D45670364138B7FCE1BE523A3284, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:51,462 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6]
2016-05-17 21:29:51,462 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6]
2016-05-17 21:29:51,462 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6]
2016-05-17 21:29:51,463 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6]. Starting root input initializers: 1
2016-05-17 21:29:51,463 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:51,463 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:51,463 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC]
2016-05-17 21:29:51,463 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_RESULTS, applicationId=application_1463520591303_0001, dagName=[1FE18A7514BE48F5BEA72795AD8B281D/CE3D68EE69F542D6A7A68CBC22D1E3DF] every/(1/1) ...atform/ALL_VALUES_RESULTS
2016-05-17 21:29:51,465 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:51,465 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC]
2016-05-17 21:29:51,466 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC]
2016-05-17 21:29:51,466 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC]
2016-05-17 21:29:51,466 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=E6C985E38DB84CC096B110E6AF138B50, vertexId=vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:51,466 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=E6C985E38DB84CC096B110E6AF138B50, vertexId=vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC]
2016-05-17 21:29:51,467 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=E6C985E38DB84CC096B110E6AF138B50, vertexName=F36BAD8E92DF441CBCB6172E852450BC, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:51,467 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=E6C985E38DB84CC096B110E6AF138B50, vertexId=vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC]
2016-05-17 21:29:51,468 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:51,468 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F36BAD8E92DF441CBCB6172E852450BC, vertexId=vertex_1463520591303_0001_1_01, initRequestedTime=1463520591463, initedTime=1463520591468, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:51,469 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:51,469 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: D9B7D45670364138B7FCE1BE523A3284 on vertex vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6]
2016-05-17 21:29:51,471 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:51,471 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input D9B7D45670364138B7FCE1BE523A3284 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:51,472 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:51,472 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:51,473 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:51,474 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: D9B7D45670364138B7FCE1BE523A3284 on vertex vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6]
2016-05-17 21:29:51,474 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:51,474 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {D9B7D45670364138B7FCE1BE523A3284=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:51,474 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6] parallelism set to 1
2016-05-17 21:29:51,474 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:51,474 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520591303_0001_1_00, eventCount=1
2016-05-17 21:29:51,475 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C492EE4B7C1047E0818C6C2A356AA5E6, vertexId=vertex_1463520591303_0001_1_00, initRequestedTime=1463520591462, initedTime=1463520591475, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:51,475 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6] with distanceFromRoot: 0
2016-05-17 21:29:51,475 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:51,475 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:51,475 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C492EE4B7C1047E0818C6C2A356AA5E6
2016-05-17 21:29:51,478 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6]
2016-05-17 21:29:51,478 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:51,479 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520591303_0001_1_00, startRequestedTime=1463520591475, startedTime=1463520591475
2016-05-17 21:29:51,479 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:51,479 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:51,478 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520591303_0001, with dag id: dag_1463520591303_0001_1
2016-05-17 21:29:51,479 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:TASK_STARTED]: vertexName=C492EE4B7C1047E0818C6C2A356AA5E6, taskId=task_1463520591303_0001_1_00_000000, scheduledTime=1463520591479, launchTime=1463520591479
2016-05-17 21:29:51,480 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520591303_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:51,480 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520591303_0001_1_00 for vertex: vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC] numStartedSources: 1 numSources: 1
2016-05-17 21:29:51,480 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC] with distanceFromRoot: 1
2016-05-17 21:29:51,481 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6]
2016-05-17 21:29:51,481 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C492EE4B7C1047E0818C6C2A356AA5E6 in vertex: F36BAD8E92DF441CBCB6172E852450BC
2016-05-17 21:29:51,481 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F36BAD8E92DF441CBCB6172E852450BC with 1 source tasks and 1 pending tasks
2016-05-17 21:29:51,481 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:51,481 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520591303_0001_1_01, startRequestedTime=1463520591481, startedTime=1463520591481
2016-05-17 21:29:51,482 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:51,482 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1FE18A7514BE48F5BEA72795AD8B281D/CE3D68EE69F542D6A7A68CBC22D1E3DF] every/(1/1) ...atform/ALL_VALUES_RESULTS, VertexName: C492EE4B7C1047E0818C6C2A356AA5E6, VertexParallelism: 1, TaskAttemptID:attempt_1463520591303_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D9B7D45670364138B7FCE1BE523A3284, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F36BAD8E92DF441CBCB6172E852450BC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:51,482 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520591303_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:51,482 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:51,483 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:51,483 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:51,483 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:51,484 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520591303_0001_00_000001
2016-05-17 21:29:51,484 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591303_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:51,485 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520591303_0001_00_000001
2016-05-17 21:29:51,485 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:51,486 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:51,486 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591303_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:51,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520591303_0001_00_000001, launchTime=1463520591486
2016-05-17 21:29:51,487 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520591303_0001_00_000001
2016-05-17 21:29:51,487 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520591303_0001_1_00_000000_0] to container: [container_1463520591303_0001_00_000001]
2016-05-17 21:29:51,487 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591303_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:51,488 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520591303_0001_1_00_000000_0] started. Is using containerId: [container_1463520591303_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:51,488 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C492EE4B7C1047E0818C6C2A356AA5E6, taskAttemptId=attempt_1463520591303_0001_1_00_000000_0, startTime=1463520591488, containerId=container_1463520591303_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520591303_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:51,488 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520591303_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:51,488 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520591303_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:51,488 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:51,489 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520591303_0001_00_000001 given task: attempt_1463520591303_0001_1_00_000000_0
2016-05-17 21:29:51,489 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520591303_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520591303_0001_1_00_000000_0
2016-05-17 21:29:51 Starting to run new task attempt: attempt_1463520591303_0001_1_00_000000_0
2016-05-17 21:29:51,489 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520591303_0001_1_00_000000_0
2016-05-17 21:29:51,489 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:51,489 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:51,489 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:51,489 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:51,489 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:51,489 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:51,490 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1FE18A7514BE48F5BEA72795AD8B281D/CE3D68EE69F542D6A7A68CBC22D1E3DF] every/(1/1) ...atform/ALL_VALUES_RESULTS, VertexName: C492EE4B7C1047E0818C6C2A356AA5E6, VertexParallelism: 1, TaskAttemptID:attempt_1463520591303_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D9B7D45670364138B7FCE1BE523A3284, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F36BAD8E92DF441CBCB6172E852450BC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:51,490 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:51,490 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520591303_0001_1_00_000000_0
2016-05-17 21:29:51,490 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:51,490 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:51,491 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520591303_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:51,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:51,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D9B7D45670364138B7FCE1BE523A3284, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:51,493 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:51,493 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D9B7D45670364138B7FCE1BE523A3284
2016-05-17 21:29:51,493 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: D9B7D45670364138B7FCE1BE523A3284
2016-05-17 21:29:51,494 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:51,495 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:51,495 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: D9B7D45670364138B7FCE1BE523A3284
2016-05-17 21:29:51,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D9B7D45670364138B7FCE1BE523A3284
2016-05-17 21:29:51,496 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F36BAD8E92DF441CBCB6172E852450BC, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:51,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:51,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F36BAD8E92DF441CBCB6172E852450BC
2016-05-17 21:29:51,498 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C492EE4B7C1047E0818C6C2A356AA5E6, ordinal: 0
2016-05-17 21:29:51,498 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C492EE4B7C1047E0818C6C2A356AA5E6, mem on start (mb), free: 89, total: 363, max: 672
2016-05-17 21:29:51,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:51,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:51,499 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:51,499 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:51,499 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F36BAD8E92DF441CBCB6172E852450BC
2016-05-17 21:29:51,500 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F36BAD8E92DF441CBCB6172E852450BC
2016-05-17 21:29:51,500 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:51,500 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:51,500 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:51,500 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:51,500 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:51,501 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:51,501 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:51,501 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D9B7D45670364138B7FCE1BE523A3284, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:51,501 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F36BAD8E92DF441CBCB6172E852450BC, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:51,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:51,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D9B7D45670364138B7FCE1BE523A3284 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:51,502 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:51,502 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:51,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D9B7D45670364138B7FCE1BE523A3284
2016-05-17 21:29:51,502 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:51,502 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D9B7D45670364138B7FCE1BE523A3284
2016-05-17 21:29:51,503 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:51,503 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520591303_0001_1_00_000000_0
2016-05-17 21:29:51,506 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:51,506 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: D9B7D45670364138B7FCE1BE523A3284
2016-05-17 21:29:51,507 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 24FD29333B134961B3014B0FE66E50EE
2016-05-17 21:29:51,507 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 24FD29333B134961B3014B0FE66E50EE
2016-05-17 21:29:51,508 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:51,510 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:51,511 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2212a423
2016-05-17 21:29:51,511 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:51,511 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@4241ca2d; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@6d407256; comparator=cascading.tuple.hadoop.util.TupleComparator@75d6074e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:51,512 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:51,512 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:51,512 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:51,521 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2212a423
2016-05-17 21:29:51,522 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:51,522 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:51,534 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:51,534 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:51,534 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:51,535 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:51,535 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:51,535 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:51,535 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:51,536 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C492EE4B7C1047E0818C6C2A356AA5E6, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:51,538 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520591303_0001_1_00_000000_0
2016-05-17 21:29:51,538 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C492EE4B7C1047E0818C6C2A356AA5E6, mem on close (mb), free: 186, total: 365, max: 672
2016-05-17 21:29:51,538 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:51,538 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:51,538 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:51,538 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:51,541 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:51 Completed running task attempt: attempt_1463520591303_0001_1_00_000000_0
2016-05-17 21:29:51,544 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520591303_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:51,544 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520591303_0001_1_00, eventCount=1
2016-05-17 21:29:51,544 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C492EE4B7C1047E0818C6C2A356AA5E6, taskAttemptId=attempt_1463520591303_0001_1_00_000000_0, startTime=1463520591488, finishTime=1463520591544, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520591536, Process_Duration=2, Process_End_Time=1463520591538, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:51,544 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520591303_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:51,545 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520591303_0001_1_00_000000_0
2016-05-17 21:29:51,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:TASK_FINISHED]: vertexName=C492EE4B7C1047E0818C6C2A356AA5E6, taskId=task_1463520591303_0001_1_00_000000, startTime=1463520591488, finishTime=1463520591545, timeTaken=57, status=SUCCEEDED, successfulAttemptID=attempt_1463520591303_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520591536, Process_Duration=2, Process_End_Time=1463520591538, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:51,545 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520591303_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:51,545 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6] : 1
2016-05-17 21:29:51,545 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6]
2016-05-17 21:29:51,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:VERTEX_FINISHED]: vertexName=C492EE4B7C1047E0818C6C2A356AA5E6, vertexId=vertex_1463520591303_0001_1_00, initRequestedTime=1463520591462, initedTime=1463520591475, startRequestedTime=1463520591475, startedTime=1463520591475, finishTime=1463520591545, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520591536, Process_Duration=2, Process_End_Time=1463520591538, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520591488, firstTasksToStart=[ task_1463520591303_0001_1_00_000000 ], lastTaskFinishTime=1463520591544, lastTasksToFinish=[ task_1463520591303_0001_1_00_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520591303_0001_1_00_000000 ], longestDurationTasks=[ task_1463520591303_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:51,546 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:51,546 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC] attempt: attempt_1463520591303_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:51,546 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:51,546 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F36BAD8E92DF441CBCB6172E852450BC
2016-05-17 21:29:51,546 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC]
2016-05-17 21:29:51,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520591536, Process_Duration=2, Process_End_Time=1463520591538, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:51,546 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:51,546 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:51,546 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591303_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:51,546 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:51,546 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520591303_0001_00_000001, stoppedTime=1463520591546, exitStatus=0
2016-05-17 21:29:51,547 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591303_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:51,547 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:51,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:TASK_STARTED]: vertexName=F36BAD8E92DF441CBCB6172E852450BC, taskId=task_1463520591303_0001_1_01_000000, scheduledTime=1463520591547, launchTime=1463520591547
2016-05-17 21:29:51,547 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520591303_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:51,547 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC]
2016-05-17 21:29:51,547 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:51,547 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1FE18A7514BE48F5BEA72795AD8B281D/CE3D68EE69F542D6A7A68CBC22D1E3DF] every/(1/1) ...atform/ALL_VALUES_RESULTS, VertexName: F36BAD8E92DF441CBCB6172E852450BC, VertexParallelism: 1, TaskAttemptID:attempt_1463520591303_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C492EE4B7C1047E0818C6C2A356AA5E6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E6C985E38DB84CC096B110E6AF138B50, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:51,547 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:51,547 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520591303_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:51,547 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520591303_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:51,547 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:51,548 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:51,548 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520591303_0001_00_000001
2016-05-17 21:29:51,547 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:51,548 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520591303_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:51,548 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591303_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:51,548 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520591303_0001_00_000002
2016-05-17 21:29:51,548 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591303_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:51,549 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520591303_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:51,549 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520591303_0001_00_000001, Exiting
2016-05-17 21:29:51,549 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520591303_0001_00_000001
2016-05-17 21:29:51,549 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520591303_0001_00_000001
2016-05-17 21:29:51,551 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520591303_0001_00_000002
2016-05-17 21:29:51,552 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:51,554 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520591303_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:51,554 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520591303_0001_00_000001 completed successfully
2016-05-17 21:29:51,555 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591303_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:51,555 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591303_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:51,555 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:51,555 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520591303_0001_00_000002, launchTime=1463520591555
2016-05-17 21:29:51,556 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520591303_0001_00_000002
2016-05-17 21:29:51,556 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520591303_0001_1_01_000000_0] to container: [container_1463520591303_0001_00_000002]
2016-05-17 21:29:51,556 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591303_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:51,556 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520591303_0001_1_01_000000_0] started. Is using containerId: [container_1463520591303_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:51,557 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F36BAD8E92DF441CBCB6172E852450BC, taskAttemptId=attempt_1463520591303_0001_1_01_000000_0, startTime=1463520591556, containerId=container_1463520591303_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520591303_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:51,557 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:51,557 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520591303_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:51,557 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520591303_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:51,557 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520591303_0001_00_000002 given task: attempt_1463520591303_0001_1_01_000000_0
2016-05-17 21:29:51,557 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520591303_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520591303_0001_1_01_000000_0
2016-05-17 21:29:51 Starting to run new task attempt: attempt_1463520591303_0001_1_01_000000_0
2016-05-17 21:29:51,558 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520591303_0001_1_01_000000_0
2016-05-17 21:29:51,558 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:51,558 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:51,558 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:51,558 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:51,558 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:51,558 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:51,558 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1FE18A7514BE48F5BEA72795AD8B281D/CE3D68EE69F542D6A7A68CBC22D1E3DF] every/(1/1) ...atform/ALL_VALUES_RESULTS, VertexName: F36BAD8E92DF441CBCB6172E852450BC, VertexParallelism: 1, TaskAttemptID:attempt_1463520591303_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C492EE4B7C1047E0818C6C2A356AA5E6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E6C985E38DB84CC096B110E6AF138B50, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:51,558 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:51,559 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520591303_0001_1_01_000000_0
2016-05-17 21:29:51,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:51,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:51,559 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520591303_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:51,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C492EE4B7C1047E0818C6C2A356AA5E6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:51,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:51,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E6C985E38DB84CC096B110E6AF138B50, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:51,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:51,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C492EE4B7C1047E0818C6C2A356AA5E6
2016-05-17 21:29:51,564 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:51,564 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:51,564 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C492EE4B7C1047E0818C6C2A356AA5E6
2016-05-17 21:29:51,565 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C492EE4B7C1047E0818C6C2A356AA5E6
2016-05-17 21:29:51,565 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:51,566 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:51,566 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E6C985E38DB84CC096B110E6AF138B50
2016-05-17 21:29:51,566 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:51,567 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: E6C985E38DB84CC096B110E6AF138B50
2016-05-17 21:29:51,569 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:51,569 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:51,570 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F36BAD8E92DF441CBCB6172E852450BC, ordinal: 1
2016-05-17 21:29:51,570 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F36BAD8E92DF441CBCB6172E852450BC, mem on start (mb), free: 181, total: 365, max: 672
2016-05-17 21:29:51,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:51,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:51,570 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:51,578 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:51,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E6C985E38DB84CC096B110E6AF138B50
2016-05-17 21:29:51,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:51,579 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:51,579 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:51,579 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:51,579 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:51,579 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:51,580 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:51,580 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C492EE4B7C1047E0818C6C2A356AA5E6, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:51,580 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E6C985E38DB84CC096B110E6AF138B50, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:51,580 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:51,580 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C492EE4B7C1047E0818C6C2A356AA5E6 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:51,580 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:51,580 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C492EE4B7C1047E0818C6C2A356AA5E6
2016-05-17 21:29:51,580 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:51,580 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:51,581 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:51,581 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C492EE4B7C1047E0818C6C2A356AA5E6 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:51,581 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:51,581 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:51,581 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:51,583 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C492EE4B7C1047E0818C6C2A356AA5E6
2016-05-17 21:29:51,583 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:51,584 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520591303_0001_1_01_000000_0
2016-05-17 21:29:51,586 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:51,587 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 24FD29333B134961B3014B0FE66E50EE
2016-05-17 21:29:51,587 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_RESULTS"], id: E6C985E38DB84CC096B110E6AF138B50
2016-05-17 21:29:51,587 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_RESULTS"]
2016-05-17 21:29:51,588 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 24FD29333B134961B3014B0FE66E50EE, for 1 inputs
2016-05-17 21:29:51,588 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520591303_0001_1_00_000000_0_10297, runDuration: 45842, ]
2016-05-17 21:29:51,589 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520591303_0001_1_00_000000_0_10297] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520591589, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:51,589 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C492EE4B7C1047E0818C6C2A356AA5E6
2016-05-17 21:29:51,590 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:51,590 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [C492EE4B7C1047E0818C6C2A356AA5E6] #50 in 1ms
2016-05-17 21:29:51,590 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C492EE4B7C1047E0818C6C2A356AA5E6] #50, status:false, isInterrupted:false
2016-05-17 21:29:51,591 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:51,592 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:51,592 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:51,592 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:51,592 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:51,592 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:51,593 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C492EE4B7C1047E0818C6C2A356AA5E6
2016-05-17 21:29:51,593 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:51,593 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F36BAD8E92DF441CBCB6172E852450BC, all 1 inputs ready in: 00:00:00.005
2016-05-17 21:29:51,594 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:51,595 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@625af9b; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@142bec44
2016-05-17 21:29:51,595 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:51,599 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:51,602 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:51,602 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:51,602 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520591303_0001_1_01_000000_0
2016-05-17 21:29:51,602 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520591303_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:51,603 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520591303_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:51,606 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520591593, Process_Duration=10, Process_End_Time=1463520591603, Read_Duration=0, Tuples_Read=1]]
2016-05-17 21:29:51,607 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:51,607 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F36BAD8E92DF441CBCB6172E852450BC, taskAttemptId=attempt_1463520591303_0001_1_01_000000_0, startTime=1463520591556, finishTime=1463520591606, timeTaken=50, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520591593, Process_Duration=10, Process_End_Time=1463520591603, Read_Duration=0, Tuples_Read=1
2016-05-17 21:29:51,607 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520591303_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:51,608 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520591303_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:51,608 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:51,607 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F36BAD8E92DF441CBCB6172E852450BC, mem on close (mb), free: 177, total: 365, max: 672
2016-05-17 21:29:51,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:TASK_FINISHED]: vertexName=F36BAD8E92DF441CBCB6172E852450BC, taskId=task_1463520591303_0001_1_01_000000, startTime=1463520591556, finishTime=1463520591608, timeTaken=52, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:51,608 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520591303_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:51,609 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:51,609 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC] : 1
2016-05-17 21:29:51,609 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC] because task failed: task_1463520591303_0001_1_01_000000
2016-05-17 21:29:51,609 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:51,609 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:51,609 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC]
2016-05-17 21:29:51,609 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=E6C985E38DB84CC096B110E6AF138B50, vertexId=vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC]
2016-05-17 21:29:51,611 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:51,611 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:VERTEX_FINISHED]: vertexName=F36BAD8E92DF441CBCB6172E852450BC, vertexId=vertex_1463520591303_0001_1_01, initRequestedTime=1463520591463, initedTime=1463520591468, startRequestedTime=1463520591481, startedTime=1463520591481, finishTime=1463520591610, timeTaken=129, status=FAILED, diagnostics=Task failed, taskId=task_1463520591303_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520591556, firstTasksToStart=[ task_1463520591303_0001_1_01_000000 ], lastTaskFinishTime=1463520591606, lastTasksToFinish=[ task_1463520591303_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:51,611 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:51,611 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520591303_0001_00_000002, stoppedTime=1463520591611, exitStatus=0
2016-05-17 21:29:51,611 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591303_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:51,612 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520591303_0001_1_01_000000_0 failed: true container: container_1463520591303_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:51,612 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=F36BAD8E92DF441CBCB6172E852450BC, index=1
2016-05-17 21:29:51,612 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:51,612 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:51,612 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C492EE4B7C1047E0818C6C2A356AA5E6
2016-05-17 21:29:51,612 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:51,612 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:51,613 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520591303_0001_1
2016-05-17 21:29:51,613 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520591303_0001_1_00 [C492EE4B7C1047E0818C6C2A356AA5E6]
2016-05-17 21:29:51,613 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: E6C985E38DB84CC096B110E6AF138B50 for vertex: vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC]
2016-05-17 21:29:51,613 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:51,613 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:51,614 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:51,614 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:51,614 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:51,614 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=F36BAD8E92DF441CBCB6172E852450BC, sourceVertex=C492EE4B7C1047E0818C6C2A356AA5E6
2016-05-17 21:29:51,614 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=F36BAD8E92DF441CBCB6172E852450BC, sourceVertex=E6C985E38DB84CC096B110E6AF138B50
2016-05-17 21:29:51,614 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520591303_0001_00_000002
2016-05-17 21:29:51,614 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520591303_0001_00_000002
2016-05-17 21:29:51,615 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520591303_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:51,615 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520591303_0001_00_000002 completed but with errors
2016-05-17 21:29:51,615 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520591303_0001_1, queueSize=0
2016-05-17 21:29:51,615 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:51,616 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591303_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520591303_0001_1, startTime=1463520591462, finishTime=1463520591613, timeTaken=151, status=FAILED, diagnostics=Vertex failed, vertexName=F36BAD8E92DF441CBCB6172E852450BC, vertexId=vertex_1463520591303_0001_1_01, diagnostics=[Task failed, taskId=task_1463520591303_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520591303_0001_1_01 [F36BAD8E92DF441CBCB6172E852450BC] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520591536, Process_Duration=2, Process_End_Time=1463520591538, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:51,617 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520591303_0001_1 finished with state: FAILED
2016-05-17 21:29:51,617 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520591303_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:51,618 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520591303_0001_00_000002
2016-05-17 21:29:51,618 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520591303_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:51,618 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591303_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:51 Completed Dag: dag_1463520591303_0001_1
2016-05-17 21:29:51,618 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520591303_0001_1, dagState=FAILED
2016-05-17 21:29:51,618 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520591303_0001_1_post
2016-05-17 21:29:51,619 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:51,619 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:51,619 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520591303_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:51,627 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/ALL_VALUES_RESULTS, applicationId=application_1463520591303_0001
2016-05-17 21:29:51,628 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:51,628 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:51,628 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:51,628 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:51,628 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:51,628 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:51,629 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:51,629 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:51,629 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:51,629 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:51,629 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:51,630 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:51,630 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:51,633 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1FE18A7514BE48F5BEA72795AD8B281D/CE3D68EE69F542D6A7A68CBC22D1E3DF/.tez/application_1463520591303_0001
2016-05-17 21:29:51,633 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:51,633 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...atform/ALL_VALUES_RESULTS
2016-05-17 21:29:51,634 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:51,634 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_RESULTS/_temporary
2016-05-17 21:29:51,634 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.348
2016-05-17 21:29:51,634 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_RESULTS
2016-05-17 21:29:51,655 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:51,655 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:51,687 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-05-17 21:29:51,694 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-05-17 21:29:51,695 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:51,695 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:51,695 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:51,707 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:51,711 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_ALL
2016-05-17 21:29:51,751 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:51,751 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:51,751 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_ALL"]
2016-05-17 21:29:51,751 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:51,752 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:51,752 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:51,753 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...lyplatform/ALL_VALUES_ALL
2016-05-17 21:29:51,753 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:51,765 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:51,770 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:51,770 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:51,770 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:51,779 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C00C84FC322D433CB42C8A16F91B51CF/E6E36A1700EB46CBA18752A262977468/.tez/application_1463520591770_0001 doesn't exist and is created
2016-05-17 21:29:51,794 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:51,794 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:51,795 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C00C84FC322D433CB42C8A16F91B51CF/E6E36A1700EB46CBA18752A262977468/.tez/application_1463520591770_0001
2016-05-17 21:29:51,796 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520591770_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:51,805 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:51,805 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:51,806 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:51,806 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:51,806 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C00C84FC322D433CB42C8A16F91B51CF/E6E36A1700EB46CBA18752A262977468/.tez/application_1463520591770_0001/localmode-log-dir
2016-05-17 21:29:51,806 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C00C84FC322D433CB42C8A16F91B51CF/E6E36A1700EB46CBA18752A262977468/.tez/application_1463520591770_0001/localmode-log-dir/history.txt.appattempt_1463520591770_0001_000000, maxErrors=10
2016-05-17 21:29:51,807 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:51,807 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520591770_0001_000000, appSubmitTime=1463520591796, launchTime=1463520591796
2016-05-17 21:29:51,810 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:51,810 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:51,811 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:51,811 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:51,813 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:42467
2016-05-17 21:29:51,814 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:51,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520591770_0001_000000, startTime=1463520591815
2016-05-17 21:29:51,815 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:51,815 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:51,894 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:51,894 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:51,895 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_VALUES_ALL, applicationId=application_1463520591770_0001, dagName=[C00C84FC322D433CB42C8A16F91B51CF/E6E36A1700EB46CBA18752A262977468] every/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-05-17 21:29:51,908 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C00C84FC322D433CB42C8A16F91B51CF/E6E36A1700EB46CBA18752A262977468] every/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-05-17 21:29:51,910 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520591770_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C00C84FC322D433CB42C8A16F91B51CF/E6E36A1700EB46CBA18752A262977468/.tez/application_1463520591770_0001/localmode-log-dir/dag_1463520591770_0001_1.dot
2016-05-17 21:29:51,910 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C00C84FC322D433CB42C8A16F91B51CF/E6E36A1700EB46CBA18752A262977468/.tez/application_1463520591770_0001/localmode-log-dir/dag_1463520591770_0001_1-tez-dag.pb.txt
2016-05-17 21:29:51,915 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520591770_0001_1
2016-05-17 21:29:51,916 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:51,916 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C00C84FC322D433CB42C8A16F91B51CF/E6E36A1700EB46CBA18752A262977468] every/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-05-17 21:29:51 Running Dag: dag_1463520591770_0001_1
2016-05-17 21:29:51,927 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520591770_0001_1, submitTime=1463520591909
2016-05-17 21:29:51,927 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:51,928 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C2CFE94B7F5A47A584C568E458A2364D
2016-05-17 21:29:51,928 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:51,929 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:51,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520591770_0001_1, initTime=1463520591927
2016-05-17 21:29:51,929 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520591770_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:51,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:DAG_STARTED]: dagID=dag_1463520591770_0001_1, startTime=1463520591929
2016-05-17 21:29:51,929 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:51,929 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520591770_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:51,929 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C12ED9B771874C32847A5A877CD13F9C : {07917966297549F7867ADD10F3A42E3C={InputName=07917966297549F7867ADD10F3A42E3C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:51,929 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 07917966297549F7867ADD10F3A42E3C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:51,929 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C]
2016-05-17 21:29:51,929 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C]
2016-05-17 21:29:51,930 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C]
2016-05-17 21:29:51,930 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C]. Starting root input initializers: 1
2016-05-17 21:29:51,930 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:51,930 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D]
2016-05-17 21:29:51,930 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:51,932 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 07917966297549F7867ADD10F3A42E3C on vertex vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C]
2016-05-17 21:29:51,933 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:51,933 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D]
2016-05-17 21:29:51,933 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D]
2016-05-17 21:29:51,933 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_VALUES_ALL, applicationId=application_1463520591770_0001, dagName=[C00C84FC322D433CB42C8A16F91B51CF/E6E36A1700EB46CBA18752A262977468] every/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-05-17 21:29:51,933 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D]
2016-05-17 21:29:51,934 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=23B3FF62B76F41E4BB37C72CD76E3A6D, vertexId=vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:51,934 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=23B3FF62B76F41E4BB37C72CD76E3A6D, vertexId=vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D]
2016-05-17 21:29:51,933 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:51,934 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 07917966297549F7867ADD10F3A42E3C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:51,935 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:51,935 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:51,936 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=23B3FF62B76F41E4BB37C72CD76E3A6D, vertexName=C2CFE94B7F5A47A584C568E458A2364D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:51,936 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=23B3FF62B76F41E4BB37C72CD76E3A6D, vertexId=vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D]
2016-05-17 21:29:51,937 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:51,937 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 07917966297549F7867ADD10F3A42E3C on vertex vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C]
2016-05-17 21:29:51,937 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C2CFE94B7F5A47A584C568E458A2364D, vertexId=vertex_1463520591770_0001_1_01, initRequestedTime=1463520591930, initedTime=1463520591937, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:51,937 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:51,938 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:51,938 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:51,939 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {07917966297549F7867ADD10F3A42E3C=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:51,939 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C] parallelism set to 1
2016-05-17 21:29:51,939 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:51,939 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520591770_0001_1_00, eventCount=1
2016-05-17 21:29:51,940 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:51,940 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C12ED9B771874C32847A5A877CD13F9C, vertexId=vertex_1463520591770_0001_1_00, initRequestedTime=1463520591929, initedTime=1463520591940, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:51,940 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C] with distanceFromRoot: 0
2016-05-17 21:29:51,940 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:51,940 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C12ED9B771874C32847A5A877CD13F9C
2016-05-17 21:29:51,941 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C]
2016-05-17 21:29:51,941 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:51,941 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520591770_0001_1_00, startRequestedTime=1463520591940, startedTime=1463520591940
2016-05-17 21:29:51,941 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:51,942 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:51,942 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:TASK_STARTED]: vertexName=C12ED9B771874C32847A5A877CD13F9C, taskId=task_1463520591770_0001_1_00_000000, scheduledTime=1463520591942, launchTime=1463520591942
2016-05-17 21:29:51,942 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520591770_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:51,943 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520591770_0001_1_00 for vertex: vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D] numStartedSources: 1 numSources: 1
2016-05-17 21:29:51,943 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D] with distanceFromRoot: 1
2016-05-17 21:29:51,943 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C]
2016-05-17 21:29:51,943 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C12ED9B771874C32847A5A877CD13F9C in vertex: C2CFE94B7F5A47A584C568E458A2364D
2016-05-17 21:29:51,943 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C2CFE94B7F5A47A584C568E458A2364D with 1 source tasks and 1 pending tasks
2016-05-17 21:29:51,944 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:51,944 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520591770_0001_1_01, startRequestedTime=1463520591943, startedTime=1463520591943
2016-05-17 21:29:51,944 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:51,944 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C00C84FC322D433CB42C8A16F91B51CF/E6E36A1700EB46CBA18752A262977468] every/(1/1) ...lyplatform/ALL_VALUES_ALL, VertexName: C12ED9B771874C32847A5A877CD13F9C, VertexParallelism: 1, TaskAttemptID:attempt_1463520591770_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=07917966297549F7867ADD10F3A42E3C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C2CFE94B7F5A47A584C568E458A2364D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:51,945 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520591770_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:51,945 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:51,946 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:51,946 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:51,946 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:51,947 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520591770_0001_00_000001
2016-05-17 21:29:51,947 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591770_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:51,948 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520591770_0001_00_000001
2016-05-17 21:29:51,948 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:51,949 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:51,949 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591770_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:51,949 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520591770_0001_00_000001, launchTime=1463520591949
2016-05-17 21:29:51,950 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520591770_0001_00_000001
2016-05-17 21:29:51,950 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520591770_0001_1_00_000000_0] to container: [container_1463520591770_0001_00_000001]
2016-05-17 21:29:51,950 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591770_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:51,950 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520591770_0001_00_000001 given task: attempt_1463520591770_0001_1_00_000000_0
2016-05-17 21:29:51,950 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520591770_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520591770_0001_1_00_000000_0
2016-05-17 21:29:51,951 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520591770_0001_1_00_000000_0] started. Is using containerId: [container_1463520591770_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:51,951 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C12ED9B771874C32847A5A877CD13F9C, taskAttemptId=attempt_1463520591770_0001_1_00_000000_0, startTime=1463520591951, containerId=container_1463520591770_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520591770_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:51,951 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520591770_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:51,951 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:51,951 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520591770_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:51 Starting to run new task attempt: attempt_1463520591770_0001_1_00_000000_0
2016-05-17 21:29:51,951 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520591770_0001_1_00_000000_0
2016-05-17 21:29:51,951 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:51,952 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:51,952 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:51,952 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:51,952 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:51,952 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:51,952 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C00C84FC322D433CB42C8A16F91B51CF/E6E36A1700EB46CBA18752A262977468] every/(1/1) ...lyplatform/ALL_VALUES_ALL, VertexName: C12ED9B771874C32847A5A877CD13F9C, VertexParallelism: 1, TaskAttemptID:attempt_1463520591770_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=07917966297549F7867ADD10F3A42E3C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C2CFE94B7F5A47A584C568E458A2364D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:51,953 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:51,953 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520591770_0001, with dag id: dag_1463520591770_0001_1
2016-05-17 21:29:51,953 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520591770_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:51,953 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520591770_0001_1_00_000000_0
2016-05-17 21:29:51,954 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:51,954 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:51,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:51,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=07917966297549F7867ADD10F3A42E3C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:51,957 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:51,957 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 07917966297549F7867ADD10F3A42E3C
2016-05-17 21:29:51,958 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C2CFE94B7F5A47A584C568E458A2364D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:51,958 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:51,958 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 07917966297549F7867ADD10F3A42E3C
2016-05-17 21:29:51,960 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:51,960 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 07917966297549F7867ADD10F3A42E3C
2016-05-17 21:29:51,960 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 07917966297549F7867ADD10F3A42E3C
2016-05-17 21:29:51,960 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:51,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C2CFE94B7F5A47A584C568E458A2364D
2016-05-17 21:29:51,962 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C12ED9B771874C32847A5A877CD13F9C, ordinal: 0
2016-05-17 21:29:51,962 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C12ED9B771874C32847A5A877CD13F9C, mem on start (mb), free: 103, total: 365, max: 672
2016-05-17 21:29:51,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:51,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:51,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:51,963 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:51,963 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C2CFE94B7F5A47A584C568E458A2364D
2016-05-17 21:29:51,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C2CFE94B7F5A47A584C568E458A2364D
2016-05-17 21:29:51,964 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:51,964 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:51,964 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:51,964 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:51,965 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:51,965 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:51,965 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:51,965 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 07917966297549F7867ADD10F3A42E3C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:51,966 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C2CFE94B7F5A47A584C568E458A2364D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:51,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:51,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 07917966297549F7867ADD10F3A42E3C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:51,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:51,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:51,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 07917966297549F7867ADD10F3A42E3C
2016-05-17 21:29:51,966 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:51,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 07917966297549F7867ADD10F3A42E3C
2016-05-17 21:29:51,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:51,967 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520591770_0001_1_00_000000_0
2016-05-17 21:29:51,967 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:51,967 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@c560c27
2016-05-17 21:29:51,968 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@c560c27
2016-05-17 21:29:51,968 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:51,968 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:51,970 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:51,971 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 07917966297549F7867ADD10F3A42E3C
2016-05-17 21:29:51,971 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 04D4B1AC911D4B3DB6DD43B0E6491A29
2016-05-17 21:29:51,971 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 04D4B1AC911D4B3DB6DD43B0E6491A29
2016-05-17 21:29:51,973 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:51,976 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:51,976 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@71637d4d; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@3ab78463; comparator=cascading.tuple.hadoop.util.TupleComparator@66549db2; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:51,976 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:51,976 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:51,976 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:51,998 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:51,999 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:51,999 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:51,999 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:51,999 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:52,000 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:52,001 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:52,002 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C12ED9B771874C32847A5A877CD13F9C, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:52,004 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520591770_0001_1_00_000000_0
2016-05-17 21:29:52,004 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C12ED9B771874C32847A5A877CD13F9C, mem on close (mb), free: 186, total: 365, max: 672
2016-05-17 21:29:52,004 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:52,004 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:52,005 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:52,005 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:52,008 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:52 Completed running task attempt: attempt_1463520591770_0001_1_00_000000_0
2016-05-17 21:29:52,011 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520591770_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:52,011 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520591770_0001_1_00, eventCount=1
2016-05-17 21:29:52,012 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C12ED9B771874C32847A5A877CD13F9C, taskAttemptId=attempt_1463520591770_0001_1_00_000000_0, startTime=1463520591951, finishTime=1463520592012, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520592002, Process_Duration=2, Process_End_Time=1463520592004, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:52,012 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520591770_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:52,012 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520591770_0001_1_00_000000_0
2016-05-17 21:29:52,012 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:52,012 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:TASK_FINISHED]: vertexName=C12ED9B771874C32847A5A877CD13F9C, taskId=task_1463520591770_0001_1_00_000000, startTime=1463520591951, finishTime=1463520592012, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1463520591770_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520592002, Process_Duration=2, Process_End_Time=1463520592004, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:52,013 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520591770_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:52,013 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:52,013 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C] : 1
2016-05-17 21:29:52,013 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C]
2016-05-17 21:29:52,013 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:52,013 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:VERTEX_FINISHED]: vertexName=C12ED9B771874C32847A5A877CD13F9C, vertexId=vertex_1463520591770_0001_1_00, initRequestedTime=1463520591929, initedTime=1463520591940, startRequestedTime=1463520591940, startedTime=1463520591940, finishTime=1463520592013, timeTaken=73, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520592002, Process_Duration=2, Process_End_Time=1463520592004, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520591951, firstTasksToStart=[ task_1463520591770_0001_1_00_000000 ], lastTaskFinishTime=1463520592012, lastTasksToFinish=[ task_1463520591770_0001_1_00_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520591770_0001_1_00_000000 ], longestDurationTasks=[ task_1463520591770_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:52,013 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:52,013 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:52,013 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D] attempt: attempt_1463520591770_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:52,014 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C2CFE94B7F5A47A584C568E458A2364D
2016-05-17 21:29:52,014 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D]
2016-05-17 21:29:52,014 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591770_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:52,014 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520591770_0001_00_000001, stoppedTime=1463520592014, exitStatus=0
2016-05-17 21:29:52,014 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591770_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:52,014 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:52,014 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:TASK_STARTED]: vertexName=C2CFE94B7F5A47A584C568E458A2364D, taskId=task_1463520591770_0001_1_01_000000, scheduledTime=1463520592014, launchTime=1463520592014
2016-05-17 21:29:52,015 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520591770_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:52,015 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D]
2016-05-17 21:29:52,015 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C00C84FC322D433CB42C8A16F91B51CF/E6E36A1700EB46CBA18752A262977468] every/(1/1) ...lyplatform/ALL_VALUES_ALL, VertexName: C2CFE94B7F5A47A584C568E458A2364D, VertexParallelism: 1, TaskAttemptID:attempt_1463520591770_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C12ED9B771874C32847A5A877CD13F9C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=23B3FF62B76F41E4BB37C72CD76E3A6D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:52,015 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520591770_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:52,015 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:52,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520592002, Process_Duration=2, Process_End_Time=1463520592004, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:52,015 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520591770_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:52,016 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591770_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:52,016 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:52,016 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:52,016 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:52,016 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:52,016 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520591770_0001_00_000001
2016-05-17 21:29:52,016 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520591770_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:52,017 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520591770_0001_00_000002
2016-05-17 21:29:52,017 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591770_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:52,017 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520591770_0001_00_000001: 1 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:52,017 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520591770_0001_00_000001, Exiting
2016-05-17 21:29:52,017 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520591770_0001_00_000001
2016-05-17 21:29:52,017 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520591770_0001_00_000001
2016-05-17 21:29:52,018 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520591770_0001_00_000002
2016-05-17 21:29:52,019 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:52,020 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520591770_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:52,020 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520591770_0001_00_000001 completed successfully
2016-05-17 21:29:52,021 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591770_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:52,021 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591770_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:52,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:52,021 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520591770_0001_00_000002, launchTime=1463520592021
2016-05-17 21:29:52,022 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520591770_0001_00_000002
2016-05-17 21:29:52,023 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520591770_0001_1_01_000000_0] to container: [container_1463520591770_0001_00_000002]
2016-05-17 21:29:52,023 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591770_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:52,023 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520591770_0001_1_01_000000_0] started. Is using containerId: [container_1463520591770_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:52,023 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C2CFE94B7F5A47A584C568E458A2364D, taskAttemptId=attempt_1463520591770_0001_1_01_000000_0, startTime=1463520592023, containerId=container_1463520591770_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520591770_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:52,023 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:52,023 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520591770_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:52,024 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520591770_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:52,024 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520591770_0001_00_000002 given task: attempt_1463520591770_0001_1_01_000000_0
2016-05-17 21:29:52,024 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520591770_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520591770_0001_1_01_000000_0
2016-05-17 21:29:52 Starting to run new task attempt: attempt_1463520591770_0001_1_01_000000_0
2016-05-17 21:29:52,024 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520591770_0001_1_01_000000_0
2016-05-17 21:29:52,024 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:52,024 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:52,025 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:52,025 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:52,025 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:52,025 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:52,025 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C00C84FC322D433CB42C8A16F91B51CF/E6E36A1700EB46CBA18752A262977468] every/(1/1) ...lyplatform/ALL_VALUES_ALL, VertexName: C2CFE94B7F5A47A584C568E458A2364D, VertexParallelism: 1, TaskAttemptID:attempt_1463520591770_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C12ED9B771874C32847A5A877CD13F9C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=23B3FF62B76F41E4BB37C72CD76E3A6D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:52,025 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:52,026 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520591770_0001_1_01_000000_0
2016-05-17 21:29:52,026 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:52,026 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520591770_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:52,026 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:52,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C12ED9B771874C32847A5A877CD13F9C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:52,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=23B3FF62B76F41E4BB37C72CD76E3A6D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:52,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:52,027 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:52,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C12ED9B771874C32847A5A877CD13F9C
2016-05-17 21:29:52,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:52,028 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 23B3FF62B76F41E4BB37C72CD76E3A6D
2016-05-17 21:29:52,028 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:52,029 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 23B3FF62B76F41E4BB37C72CD76E3A6D
2016-05-17 21:29:52,029 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:52,029 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:52,029 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C12ED9B771874C32847A5A877CD13F9C
2016-05-17 21:29:52,029 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C12ED9B771874C32847A5A877CD13F9C
2016-05-17 21:29:52,031 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:52,035 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C2CFE94B7F5A47A584C568E458A2364D, ordinal: 1
2016-05-17 21:29:52,035 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C2CFE94B7F5A47A584C568E458A2364D, mem on start (mb), free: 180, total: 365, max: 672
2016-05-17 21:29:52,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:52,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:52,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:52,038 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:52,038 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 23B3FF62B76F41E4BB37C72CD76E3A6D
2016-05-17 21:29:52,038 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:52,038 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:52,038 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:52,038 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:52,038 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:52,038 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:52,038 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:52,039 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 23B3FF62B76F41E4BB37C72CD76E3A6D, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:52,039 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C12ED9B771874C32847A5A877CD13F9C, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:52,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:52,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C12ED9B771874C32847A5A877CD13F9C being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:52,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C12ED9B771874C32847A5A877CD13F9C
2016-05-17 21:29:52,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:52,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:52,039 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:52,039 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:52,040 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C12ED9B771874C32847A5A877CD13F9C with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:52,040 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:52,040 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:52,040 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:52,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C12ED9B771874C32847A5A877CD13F9C
2016-05-17 21:29:52,040 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:52,041 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520591770_0001_1_01_000000_0
2016-05-17 21:29:52,043 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:52,043 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 04D4B1AC911D4B3DB6DD43B0E6491A29
2016-05-17 21:29:52,043 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_ALL"], id: 23B3FF62B76F41E4BB37C72CD76E3A6D
2016-05-17 21:29:52,043 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_ALL"]
2016-05-17 21:29:52,043 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 04D4B1AC911D4B3DB6DD43B0E6491A29, for 1 inputs
2016-05-17 21:29:52,044 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520591770_0001_1_00_000000_0_10303, runDuration: 48794, ]
2016-05-17 21:29:52,045 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520591770_0001_1_00_000000_0_10303] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520592045, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:52,045 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C12ED9B771874C32847A5A877CD13F9C
2016-05-17 21:29:52,046 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:52,046 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [C12ED9B771874C32847A5A877CD13F9C] #51 in 1ms
2016-05-17 21:29:52,046 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C12ED9B771874C32847A5A877CD13F9C] #51, status:false, isInterrupted:false
2016-05-17 21:29:52,046 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:52,047 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:52,047 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:52,047 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:52,047 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:52,048 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:52,048 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C12ED9B771874C32847A5A877CD13F9C
2016-05-17 21:29:52,048 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:52,048 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C2CFE94B7F5A47A584C568E458A2364D, all 1 inputs ready in: 00:00:00.004
2016-05-17 21:29:52,049 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:52,050 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@5a65f83f; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@5c8d1e76
2016-05-17 21:29:52,051 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:52,051 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:52,051 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:52,052 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:52,052 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520591770_0001_1_01_000000_0
2016-05-17 21:29:52,052 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520591770_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:52,053 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520591770_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:52,054 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:52,054 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C2CFE94B7F5A47A584C568E458A2364D, taskAttemptId=attempt_1463520591770_0001_1_01_000000_0, startTime=1463520592023, finishTime=1463520592054, timeTaken=31, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520592048, Process_Duration=5, Process_End_Time=1463520592053, Read_Duration=1, Tuples_Read=1
2016-05-17 21:29:52,054 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520591770_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:52,054 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520591770_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:52,054 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:52,054 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:TASK_FINISHED]: vertexName=C2CFE94B7F5A47A584C568E458A2364D, taskId=task_1463520591770_0001_1_01_000000, startTime=1463520592023, finishTime=1463520592054, timeTaken=31, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:52,056 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520591770_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:52,056 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:52,056 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D] : 1
2016-05-17 21:29:52,056 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D] because task failed: task_1463520591770_0001_1_01_000000
2016-05-17 21:29:52,056 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:52,057 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:52,057 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D]
2016-05-17 21:29:52,057 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=23B3FF62B76F41E4BB37C72CD76E3A6D, vertexId=vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D]
2016-05-17 21:29:52,058 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520592048, Process_Duration=5, Process_End_Time=1463520592053, Read_Duration=1, Tuples_Read=1]]
2016-05-17 21:29:52,058 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:VERTEX_FINISHED]: vertexName=C2CFE94B7F5A47A584C568E458A2364D, vertexId=vertex_1463520591770_0001_1_01, initRequestedTime=1463520591930, initedTime=1463520591937, startRequestedTime=1463520591943, startedTime=1463520591943, finishTime=1463520592058, timeTaken=115, status=FAILED, diagnostics=Task failed, taskId=task_1463520591770_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520592023, firstTasksToStart=[ task_1463520591770_0001_1_01_000000 ], lastTaskFinishTime=1463520592054, lastTasksToFinish=[ task_1463520591770_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:52,058 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:52,058 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:52,059 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520591770_0001_00_000002, stoppedTime=1463520592059, exitStatus=0
2016-05-17 21:29:52,058 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C2CFE94B7F5A47A584C568E458A2364D, mem on close (mb), free: 176, total: 365, max: 672
2016-05-17 21:29:52,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=C2CFE94B7F5A47A584C568E458A2364D, index=1
2016-05-17 21:29:52,059 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591770_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:52,060 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C12ED9B771874C32847A5A877CD13F9C
2016-05-17 21:29:52,060 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520591770_0001_1_01_000000_0 failed: true container: container_1463520591770_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:52,059 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:52,060 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:52,060 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:52,060 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:52,060 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520591770_0001_1
2016-05-17 21:29:52,060 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 23B3FF62B76F41E4BB37C72CD76E3A6D for vertex: vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D]
2016-05-17 21:29:52,061 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:52,061 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:52,062 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:52,062 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:52,062 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:52,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=C2CFE94B7F5A47A584C568E458A2364D, sourceVertex=C12ED9B771874C32847A5A877CD13F9C
2016-05-17 21:29:52,062 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=C2CFE94B7F5A47A584C568E458A2364D, sourceVertex=23B3FF62B76F41E4BB37C72CD76E3A6D
2016-05-17 21:29:52,062 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520591770_0001_1_00 [C12ED9B771874C32847A5A877CD13F9C]
2016-05-17 21:29:52,063 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520591770_0001_00_000002
2016-05-17 21:29:52,063 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520591770_0001_00_000002
2016-05-17 21:29:52,063 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520591770_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:52,063 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520591770_0001_00_000002 completed but with errors
2016-05-17 21:29:52,064 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520591770_0001_1, queueSize=0
2016-05-17 21:29:52,064 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:52,064 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520591770_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520591770_0001_1, startTime=1463520591929, finishTime=1463520592063, timeTaken=134, status=FAILED, diagnostics=Vertex failed, vertexName=C2CFE94B7F5A47A584C568E458A2364D, vertexId=vertex_1463520591770_0001_1_01, diagnostics=[Task failed, taskId=task_1463520591770_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520591770_0001_1_01 [C2CFE94B7F5A47A584C568E458A2364D] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520592002, Process_Duration=2, Process_End_Time=1463520592004, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:52,064 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520591770_0001_1 finished with state: FAILED
2016-05-17 21:29:52,065 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520591770_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:52,065 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520591770_0001_00_000002
2016-05-17 21:29:52,065 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520591770_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:52,065 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520591770_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:52 Completed Dag: dag_1463520591770_0001_1
2016-05-17 21:29:52,065 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520591770_0001_1, dagState=FAILED
2016-05-17 21:29:52,066 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520591770_0001_1_post
2016-05-17 21:29:52,066 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:52,066 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:52,066 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520591770_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:52,074 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/ALL_VALUES_ALL, applicationId=application_1463520591770_0001
2016-05-17 21:29:52,075 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:52,075 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:52,075 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:52,075 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:52,075 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:52,076 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:52,076 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:52,076 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:52,076 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:52,076 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:52,077 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:52,077 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:52,077 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:52,080 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/C00C84FC322D433CB42C8A16F91B51CF/E6E36A1700EB46CBA18752A262977468/.tez/application_1463520591770_0001
2016-05-17 21:29:52,081 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:52,081 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...lyplatform/ALL_VALUES_ALL
2016-05-17 21:29:52,081 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:52,082 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_ALL/_temporary
2016-05-17 21:29:52,082 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.330
2016-05-17 21:29:52,082 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_ALL
2016-05-17 21:29:52,087 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:52,087 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:52,105 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:52,106 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:52,133 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-05-17 21:29:52,139 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-05-17 21:29:52,139 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:52,140 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:52,140 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:52,155 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:52,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_REPLACE
2016-05-17 21:29:52,197 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:52,198 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:52,198 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_REPLACE"]
2016-05-17 21:29:52,198 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:52,198 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:52,198 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:52,198 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...atform/ALL_VALUES_REPLACE
2016-05-17 21:29:52,198 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:52,210 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:52,215 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:52,215 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:52,215 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:52,224 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/02856366506043D193A4276401290753/E2588E98840848BEA43240B4AF79634C/.tez/application_1463520592215_0001 doesn't exist and is created
2016-05-17 21:29:52,238 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:52,238 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:52,239 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/02856366506043D193A4276401290753/E2588E98840848BEA43240B4AF79634C/.tez/application_1463520592215_0001
2016-05-17 21:29:52,242 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520592215_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:52,253 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:52,253 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:52,254 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:52,254 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:52,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/02856366506043D193A4276401290753/E2588E98840848BEA43240B4AF79634C/.tez/application_1463520592215_0001/localmode-log-dir
2016-05-17 21:29:52,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/02856366506043D193A4276401290753/E2588E98840848BEA43240B4AF79634C/.tez/application_1463520592215_0001/localmode-log-dir/history.txt.appattempt_1463520592215_0001_000000, maxErrors=10
2016-05-17 21:29:52,255 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:52,255 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520592215_0001_000000, appSubmitTime=1463520592241, launchTime=1463520592241
2016-05-17 21:29:52,258 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:52,258 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:52,259 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:52,259 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:46632
2016-05-17 21:29:52,260 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:52,261 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:52,262 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520592215_0001_000000, startTime=1463520592262
2016-05-17 21:29:52,262 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:52,262 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:52,338 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:52,339 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:52,339 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_REPLACE, applicationId=application_1463520592215_0001, dagName=[02856366506043D193A4276401290753/E2588E98840848BEA43240B4AF79634C] every/(1/1) ...atform/ALL_VALUES_REPLACE
2016-05-17 21:29:52,353 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [02856366506043D193A4276401290753/E2588E98840848BEA43240B4AF79634C] every/(1/1) ...atform/ALL_VALUES_REPLACE
2016-05-17 21:29:52,354 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520592215_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/02856366506043D193A4276401290753/E2588E98840848BEA43240B4AF79634C/.tez/application_1463520592215_0001/localmode-log-dir/dag_1463520592215_0001_1.dot
2016-05-17 21:29:52,354 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/02856366506043D193A4276401290753/E2588E98840848BEA43240B4AF79634C/.tez/application_1463520592215_0001/localmode-log-dir/dag_1463520592215_0001_1-tez-dag.pb.txt
2016-05-17 21:29:52,360 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520592215_0001_1
2016-05-17 21:29:52,360 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:52,360 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [02856366506043D193A4276401290753/E2588E98840848BEA43240B4AF79634C] every/(1/1) ...atform/ALL_VALUES_REPLACE
2016-05-17 21:29:52 Running Dag: dag_1463520592215_0001_1
2016-05-17 21:29:52,369 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520592215_0001_1, submitTime=1463520592353
2016-05-17 21:29:52,369 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:52,370 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EB0FFF9CB14F4A2CA890569B1147A594
2016-05-17 21:29:52,370 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:52,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520592215_0001_1, initTime=1463520592369
2016-05-17 21:29:52,370 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:52,370 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520592215_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:52,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:DAG_STARTED]: dagID=dag_1463520592215_0001_1, startTime=1463520592371
2016-05-17 21:29:52,371 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:52,371 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:52,372 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520592215_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:52,372 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B9D86D6DA9744CF3B4124686C7D7B9B4 : {51E39698353F4FD38E2CA7B489EDB4DE={InputName=51E39698353F4FD38E2CA7B489EDB4DE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:52,372 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 51E39698353F4FD38E2CA7B489EDB4DE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:52,372 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4]
2016-05-17 21:29:52,372 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4]
2016-05-17 21:29:52,372 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4]
2016-05-17 21:29:52,372 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4]. Starting root input initializers: 1
2016-05-17 21:29:52,373 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:52,373 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594]
2016-05-17 21:29:52,373 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_REPLACE, applicationId=application_1463520592215_0001, dagName=[02856366506043D193A4276401290753/E2588E98840848BEA43240B4AF79634C] every/(1/1) ...atform/ALL_VALUES_REPLACE
2016-05-17 21:29:52,374 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 51E39698353F4FD38E2CA7B489EDB4DE on vertex vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4]
2016-05-17 21:29:52,375 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:52,375 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594]
2016-05-17 21:29:52,375 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594]
2016-05-17 21:29:52,375 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594]
2016-05-17 21:29:52,376 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1C708DE1101148C48240E7DCF2C115CD, vertexId=vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:52,376 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1C708DE1101148C48240E7DCF2C115CD, vertexId=vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594]
2016-05-17 21:29:52,376 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:52,376 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 51E39698353F4FD38E2CA7B489EDB4DE asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:52,376 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:52,377 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:52,377 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1C708DE1101148C48240E7DCF2C115CD, vertexName=EB0FFF9CB14F4A2CA890569B1147A594, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:52,378 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1C708DE1101148C48240E7DCF2C115CD, vertexId=vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594]
2016-05-17 21:29:52,378 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:52,378 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 51E39698353F4FD38E2CA7B489EDB4DE on vertex vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4]
2016-05-17 21:29:52,379 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:52,379 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EB0FFF9CB14F4A2CA890569B1147A594, vertexId=vertex_1463520592215_0001_1_01, initRequestedTime=1463520592373, initedTime=1463520592379, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:52,379 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:52,379 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:52,379 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {51E39698353F4FD38E2CA7B489EDB4DE=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:52,380 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4] parallelism set to 1
2016-05-17 21:29:52,380 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:52,380 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520592215_0001_1_00, eventCount=1
2016-05-17 21:29:52,380 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:52,381 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B9D86D6DA9744CF3B4124686C7D7B9B4, vertexId=vertex_1463520592215_0001_1_00, initRequestedTime=1463520592372, initedTime=1463520592380, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:52,381 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4] with distanceFromRoot: 0
2016-05-17 21:29:52,381 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:52,381 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B9D86D6DA9744CF3B4124686C7D7B9B4
2016-05-17 21:29:52,381 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4]
2016-05-17 21:29:52,381 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:52,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520592215_0001_1_00, startRequestedTime=1463520592381, startedTime=1463520592381
2016-05-17 21:29:52,382 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:52,382 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:52,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:TASK_STARTED]: vertexName=B9D86D6DA9744CF3B4124686C7D7B9B4, taskId=task_1463520592215_0001_1_00_000000, scheduledTime=1463520592382, launchTime=1463520592382
2016-05-17 21:29:52,382 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520592215_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:52,382 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520592215_0001_1_00 for vertex: vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594] numStartedSources: 1 numSources: 1
2016-05-17 21:29:52,383 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594] with distanceFromRoot: 1
2016-05-17 21:29:52,383 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4]
2016-05-17 21:29:52,383 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B9D86D6DA9744CF3B4124686C7D7B9B4 in vertex: EB0FFF9CB14F4A2CA890569B1147A594
2016-05-17 21:29:52,383 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: EB0FFF9CB14F4A2CA890569B1147A594 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:52,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520592215_0001_1_01, startRequestedTime=1463520592383, startedTime=1463520592383
2016-05-17 21:29:52,383 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:52,383 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:52,384 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [02856366506043D193A4276401290753/E2588E98840848BEA43240B4AF79634C] every/(1/1) ...atform/ALL_VALUES_REPLACE, VertexName: B9D86D6DA9744CF3B4124686C7D7B9B4, VertexParallelism: 1, TaskAttemptID:attempt_1463520592215_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=51E39698353F4FD38E2CA7B489EDB4DE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EB0FFF9CB14F4A2CA890569B1147A594, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:52,384 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520592215_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:52,384 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:52,385 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:52,385 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:52,385 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:52,386 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520592215_0001_00_000001
2016-05-17 21:29:52,386 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592215_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:52,387 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520592215_0001_00_000001
2016-05-17 21:29:52,387 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:52,388 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592215_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:52,388 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:52,388 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520592215_0001_00_000001, launchTime=1463520592388
2016-05-17 21:29:52,389 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520592215_0001_00_000001
2016-05-17 21:29:52,389 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520592215_0001_1_00_000000_0] to container: [container_1463520592215_0001_00_000001]
2016-05-17 21:29:52,389 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592215_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:52,390 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520592215_0001_00_000001 given task: attempt_1463520592215_0001_1_00_000000_0
2016-05-17 21:29:52,390 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520592215_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520592215_0001_1_00_000000_0
2016-05-17 21:29:52,390 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520592215_0001_1_00_000000_0] started. Is using containerId: [container_1463520592215_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:52,390 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B9D86D6DA9744CF3B4124686C7D7B9B4, taskAttemptId=attempt_1463520592215_0001_1_00_000000_0, startTime=1463520592390, containerId=container_1463520592215_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520592215_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:52,390 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520592215_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:52,390 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:52,390 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520592215_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:52 Starting to run new task attempt: attempt_1463520592215_0001_1_00_000000_0
2016-05-17 21:29:52,392 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520592215_0001_1_00_000000_0
2016-05-17 21:29:52,392 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:52,392 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:52,392 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:52,392 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:52,393 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:52,393 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:52,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [02856366506043D193A4276401290753/E2588E98840848BEA43240B4AF79634C] every/(1/1) ...atform/ALL_VALUES_REPLACE, VertexName: B9D86D6DA9744CF3B4124686C7D7B9B4, VertexParallelism: 1, TaskAttemptID:attempt_1463520592215_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=51E39698353F4FD38E2CA7B489EDB4DE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EB0FFF9CB14F4A2CA890569B1147A594, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:52,393 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:52,394 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520592215_0001_1_00_000000_0
2016-05-17 21:29:52,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:52,394 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520592215_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:52,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:52,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:52,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=51E39698353F4FD38E2CA7B489EDB4DE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:52,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:52,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 51E39698353F4FD38E2CA7B489EDB4DE
2016-05-17 21:29:52,396 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 51E39698353F4FD38E2CA7B489EDB4DE
2016-05-17 21:29:52,398 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520592215_0001, with dag id: dag_1463520592215_0001_1
2016-05-17 21:29:52,398 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:52,398 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:52,398 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 51E39698353F4FD38E2CA7B489EDB4DE
2016-05-17 21:29:52,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 51E39698353F4FD38E2CA7B489EDB4DE
2016-05-17 21:29:52,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EB0FFF9CB14F4A2CA890569B1147A594, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:52,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:52,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EB0FFF9CB14F4A2CA890569B1147A594
2016-05-17 21:29:52,402 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B9D86D6DA9744CF3B4124686C7D7B9B4, ordinal: 0
2016-05-17 21:29:52,402 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B9D86D6DA9744CF3B4124686C7D7B9B4, mem on start (mb), free: 103, total: 365, max: 672
2016-05-17 21:29:52,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:52,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:52,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:52,403 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:52,403 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EB0FFF9CB14F4A2CA890569B1147A594
2016-05-17 21:29:52,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EB0FFF9CB14F4A2CA890569B1147A594
2016-05-17 21:29:52,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:52,403 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:52,404 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:52,404 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:52,404 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:52,404 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:52,404 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:52,405 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 51E39698353F4FD38E2CA7B489EDB4DE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:52,405 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EB0FFF9CB14F4A2CA890569B1147A594, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:52,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:52,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 51E39698353F4FD38E2CA7B489EDB4DE being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:52,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:52,406 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:52,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 51E39698353F4FD38E2CA7B489EDB4DE
2016-05-17 21:29:52,406 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:52,406 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 51E39698353F4FD38E2CA7B489EDB4DE
2016-05-17 21:29:52,406 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:52,407 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520592215_0001_1_00_000000_0
2016-05-17 21:29:52,407 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:52,407 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1fce0946
2016-05-17 21:29:52,408 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1fce0946
2016-05-17 21:29:52,408 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:52,408 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:52,409 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:52,410 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 51E39698353F4FD38E2CA7B489EDB4DE
2016-05-17 21:29:52,410 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 03AB2421C22E40489A75F7B10B95ECD8
2016-05-17 21:29:52,410 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 03AB2421C22E40489A75F7B10B95ECD8
2016-05-17 21:29:52,411 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:52,413 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:52,413 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@31f600c7; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@dad2271; comparator=cascading.tuple.hadoop.util.TupleComparator@61b02fba; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:52,413 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:52,413 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:52,413 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:52,435 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:52,435 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:52,435 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:52,435 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:52,435 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:52,436 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:52,436 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:52,437 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B9D86D6DA9744CF3B4124686C7D7B9B4, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:52,439 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520592215_0001_1_00_000000_0
2016-05-17 21:29:52,439 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B9D86D6DA9744CF3B4124686C7D7B9B4, mem on close (mb), free: 184, total: 364, max: 672
2016-05-17 21:29:52,439 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:52,439 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:52,439 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:52,440 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:52,442 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:52 Completed running task attempt: attempt_1463520592215_0001_1_00_000000_0
2016-05-17 21:29:52,445 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520592215_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:52,446 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520592215_0001_1_00, eventCount=1
2016-05-17 21:29:52,446 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:52,446 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B9D86D6DA9744CF3B4124686C7D7B9B4, taskAttemptId=attempt_1463520592215_0001_1_00_000000_0, startTime=1463520592390, finishTime=1463520592446, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520592437, Process_Duration=2, Process_End_Time=1463520592439, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:52,446 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520592215_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:52,446 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520592215_0001_1_00_000000_0
2016-05-17 21:29:52,446 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:52,447 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:52,447 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:TASK_FINISHED]: vertexName=B9D86D6DA9744CF3B4124686C7D7B9B4, taskId=task_1463520592215_0001_1_00_000000, startTime=1463520592390, finishTime=1463520592446, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1463520592215_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520592437, Process_Duration=2, Process_End_Time=1463520592439, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:52,447 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520592215_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:52,447 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4] : 1
2016-05-17 21:29:52,447 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4]
2016-05-17 21:29:52,447 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:52,447 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:VERTEX_FINISHED]: vertexName=B9D86D6DA9744CF3B4124686C7D7B9B4, vertexId=vertex_1463520592215_0001_1_00, initRequestedTime=1463520592372, initedTime=1463520592380, startRequestedTime=1463520592381, startedTime=1463520592381, finishTime=1463520592447, timeTaken=66, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520592437, Process_Duration=2, Process_End_Time=1463520592439, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520592390, firstTasksToStart=[ task_1463520592215_0001_1_00_000000 ], lastTaskFinishTime=1463520592446, lastTasksToFinish=[ task_1463520592215_0001_1_00_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520592215_0001_1_00_000000 ], longestDurationTasks=[ task_1463520592215_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:52,448 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:52,448 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592215_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:52,448 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520592437, Process_Duration=2, Process_End_Time=1463520592439, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:52,448 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:52,449 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:52,449 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:52,448 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520592215_0001_00_000001, stoppedTime=1463520592448, exitStatus=0
2016-05-17 21:29:52,449 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592215_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:52,449 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594] attempt: attempt_1463520592215_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:52,449 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EB0FFF9CB14F4A2CA890569B1147A594
2016-05-17 21:29:52,449 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594]
2016-05-17 21:29:52,449 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:52,450 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520592215_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:52,450 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520592215_0001_00_000001
2016-05-17 21:29:52,450 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:TASK_STARTED]: vertexName=EB0FFF9CB14F4A2CA890569B1147A594, taskId=task_1463520592215_0001_1_01_000000, scheduledTime=1463520592450, launchTime=1463520592450
2016-05-17 21:29:52,450 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520592215_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:52,450 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520592215_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:52,450 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:52,450 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592215_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:52,450 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594]
2016-05-17 21:29:52,450 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [02856366506043D193A4276401290753/E2588E98840848BEA43240B4AF79634C] every/(1/1) ...atform/ALL_VALUES_REPLACE, VertexName: EB0FFF9CB14F4A2CA890569B1147A594, VertexParallelism: 1, TaskAttemptID:attempt_1463520592215_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B9D86D6DA9744CF3B4124686C7D7B9B4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1C708DE1101148C48240E7DCF2C115CD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:52,450 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520592215_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:52,451 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:52,451 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520592215_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:52,451 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520592215_0001_00_000001, Exiting
2016-05-17 21:29:52,451 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520592215_0001_00_000001
2016-05-17 21:29:52,451 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520592215_0001_00_000001
2016-05-17 21:29:52,452 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520592215_0001_00_000002
2016-05-17 21:29:52,452 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592215_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:52,452 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520592215_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:52,453 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520592215_0001_00_000001 completed successfully
2016-05-17 21:29:52,453 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592215_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:52,454 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520592215_0001_00_000002
2016-05-17 21:29:52,454 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-05-17 21:29:52,456 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:52,456 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520592215_0001_00_000002, launchTime=1463520592456
2016-05-17 21:29:52,457 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592215_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:52,457 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520592215_0001_00_000002
2016-05-17 21:29:52,457 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520592215_0001_1_01_000000_0] to container: [container_1463520592215_0001_00_000002]
2016-05-17 21:29:52,457 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592215_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:52,458 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520592215_0001_00_000002 given task: attempt_1463520592215_0001_1_01_000000_0
2016-05-17 21:29:52,458 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520592215_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520592215_0001_1_01_000000_0
2016-05-17 21:29:52,458 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520592215_0001_1_01_000000_0] started. Is using containerId: [container_1463520592215_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:52,458 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EB0FFF9CB14F4A2CA890569B1147A594, taskAttemptId=attempt_1463520592215_0001_1_01_000000_0, startTime=1463520592458, containerId=container_1463520592215_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520592215_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:52 Starting to run new task attempt: attempt_1463520592215_0001_1_01_000000_0
2016-05-17 21:29:52,458 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520592215_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:52,458 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:52,458 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520592215_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:52,458 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520592215_0001_1_01_000000_0
2016-05-17 21:29:52,459 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:52,459 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:52,459 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:52,459 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:52,459 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:52,459 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:52,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [02856366506043D193A4276401290753/E2588E98840848BEA43240B4AF79634C] every/(1/1) ...atform/ALL_VALUES_REPLACE, VertexName: EB0FFF9CB14F4A2CA890569B1147A594, VertexParallelism: 1, TaskAttemptID:attempt_1463520592215_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B9D86D6DA9744CF3B4124686C7D7B9B4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1C708DE1101148C48240E7DCF2C115CD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:52,459 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:52,460 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520592215_0001_1_01_000000_0
2016-05-17 21:29:52,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:52,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:52,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B9D86D6DA9744CF3B4124686C7D7B9B4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:52,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:52,461 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520592215_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:52,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:52,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B9D86D6DA9744CF3B4124686C7D7B9B4
2016-05-17 21:29:52,463 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:52,463 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:52,463 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B9D86D6DA9744CF3B4124686C7D7B9B4
2016-05-17 21:29:52,464 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B9D86D6DA9744CF3B4124686C7D7B9B4
2016-05-17 21:29:52,463 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1C708DE1101148C48240E7DCF2C115CD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:52,464 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:52,464 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:52,464 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1C708DE1101148C48240E7DCF2C115CD
2016-05-17 21:29:52,464 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:52,465 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1C708DE1101148C48240E7DCF2C115CD
2016-05-17 21:29:52,469 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EB0FFF9CB14F4A2CA890569B1147A594, ordinal: 1
2016-05-17 21:29:52,469 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EB0FFF9CB14F4A2CA890569B1147A594, mem on start (mb), free: 178, total: 364, max: 672
2016-05-17 21:29:52,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:52,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:52,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:52,471 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:52,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1C708DE1101148C48240E7DCF2C115CD
2016-05-17 21:29:52,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:52,471 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:52,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:52,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:52,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:52,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:52,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:52,472 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B9D86D6DA9744CF3B4124686C7D7B9B4, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:52,472 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1C708DE1101148C48240E7DCF2C115CD, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:52,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:52,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B9D86D6DA9744CF3B4124686C7D7B9B4 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:52,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:52,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:52,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B9D86D6DA9744CF3B4124686C7D7B9B4
2016-05-17 21:29:52,473 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:52,474 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:52,474 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B9D86D6DA9744CF3B4124686C7D7B9B4 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:52,474 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:52,474 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:52,474 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:52,476 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B9D86D6DA9744CF3B4124686C7D7B9B4
2016-05-17 21:29:52,476 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:52,477 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520592215_0001_1_01_000000_0
2016-05-17 21:29:52,477 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520592215_0001_1_00_000000_0_10309, runDuration: 43234, ]
2016-05-17 21:29:52,478 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520592215_0001_1_00_000000_0_10309] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520592478, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:52,478 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B9D86D6DA9744CF3B4124686C7D7B9B4
2016-05-17 21:29:52,478 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:52,479 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:52,480 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 03AB2421C22E40489A75F7B10B95ECD8
2016-05-17 21:29:52,480 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_REPLACE"], id: 1C708DE1101148C48240E7DCF2C115CD
2016-05-17 21:29:52,480 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_REPLACE"]
2016-05-17 21:29:52,481 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 03AB2421C22E40489A75F7B10B95ECD8, for 1 inputs
2016-05-17 21:29:52,482 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B9D86D6DA9744CF3B4124686C7D7B9B4] #52 in 5ms
2016-05-17 21:29:52,483 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B9D86D6DA9744CF3B4124686C7D7B9B4] #52, status:false, isInterrupted:false
2016-05-17 21:29:52,484 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:52,485 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:52,485 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:52,485 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:52,485 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:52,486 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:52,486 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EB0FFF9CB14F4A2CA890569B1147A594, all 1 inputs ready in: 00:00:00.004
2016-05-17 21:29:52,486 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B9D86D6DA9744CF3B4124686C7D7B9B4
2016-05-17 21:29:52,487 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:52,488 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@25d68836; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@2788a4a2
2016-05-17 21:29:52,488 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:52,488 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:52,489 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:52,489 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:52,489 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:52,489 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520592215_0001_1_01_000000_0
2016-05-17 21:29:52,489 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520592215_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:52,490 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520592215_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:52,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520592486, Process_Duration=4, Process_End_Time=1463520592490, Read_Duration=0, Tuples_Read=1]]
2016-05-17 21:29:52,491 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EB0FFF9CB14F4A2CA890569B1147A594, mem on close (mb), free: 175, total: 364, max: 672
2016-05-17 21:29:52,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=EB0FFF9CB14F4A2CA890569B1147A594, index=1
2016-05-17 21:29:52,492 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B9D86D6DA9744CF3B4124686C7D7B9B4
2016-05-17 21:29:52,493 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:52,493 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:52,493 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EB0FFF9CB14F4A2CA890569B1147A594, taskAttemptId=attempt_1463520592215_0001_1_01_000000_0, startTime=1463520592458, finishTime=1463520592493, timeTaken=35, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520592486, Process_Duration=4, Process_End_Time=1463520592490, Read_Duration=0, Tuples_Read=1
2016-05-17 21:29:52,494 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520592215_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:52,494 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520592215_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:52,494 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:TASK_FINISHED]: vertexName=EB0FFF9CB14F4A2CA890569B1147A594, taskId=task_1463520592215_0001_1_01_000000, startTime=1463520592458, finishTime=1463520592494, timeTaken=36, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:52,494 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520592215_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:52,494 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:52,494 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:52,494 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594] : 1
2016-05-17 21:29:52,494 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594] because task failed: task_1463520592215_0001_1_01_000000
2016-05-17 21:29:52,495 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:52,495 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:52,495 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594]
2016-05-17 21:29:52,495 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=1C708DE1101148C48240E7DCF2C115CD, vertexId=vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594]
2016-05-17 21:29:52,496 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:52,496 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:52,496 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:52,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:52,496 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:VERTEX_FINISHED]: vertexName=EB0FFF9CB14F4A2CA890569B1147A594, vertexId=vertex_1463520592215_0001_1_01, initRequestedTime=1463520592373, initedTime=1463520592379, startRequestedTime=1463520592383, startedTime=1463520592383, finishTime=1463520592496, timeTaken=113, status=FAILED, diagnostics=Task failed, taskId=task_1463520592215_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520592458, firstTasksToStart=[ task_1463520592215_0001_1_01_000000 ], lastTaskFinishTime=1463520592493, lastTasksToFinish=[ task_1463520592215_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:52,496 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:52,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:52,497 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:52,497 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:52,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=EB0FFF9CB14F4A2CA890569B1147A594, sourceVertex=B9D86D6DA9744CF3B4124686C7D7B9B4
2016-05-17 21:29:52,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=EB0FFF9CB14F4A2CA890569B1147A594, sourceVertex=1C708DE1101148C48240E7DCF2C115CD
2016-05-17 21:29:52,496 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520592215_0001_00_000002, stoppedTime=1463520592496, exitStatus=0
2016-05-17 21:29:52,498 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592215_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:52,497 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520592215_0001_00_000002
2016-05-17 21:29:52,498 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520592215_0001_00_000002
2016-05-17 21:29:52,498 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520592215_0001_1_01_000000_0 failed: true container: container_1463520592215_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:52,498 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:52,498 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:52,498 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520592215_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:52,499 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520592215_0001_00_000002 completed but with errors
2016-05-17 21:29:52,499 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520592215_0001_1
2016-05-17 21:29:52,499 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520592215_0001_1_00 [B9D86D6DA9744CF3B4124686C7D7B9B4]
2016-05-17 21:29:52,499 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 1C708DE1101148C48240E7DCF2C115CD for vertex: vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594]
2016-05-17 21:29:52,500 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520592215_0001_1, queueSize=0
2016-05-17 21:29:52,500 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:52,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592215_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520592215_0001_1, startTime=1463520592371, finishTime=1463520592499, timeTaken=128, status=FAILED, diagnostics=Vertex failed, vertexName=EB0FFF9CB14F4A2CA890569B1147A594, vertexId=vertex_1463520592215_0001_1_01, diagnostics=[Task failed, taskId=task_1463520592215_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520592215_0001_1_01 [EB0FFF9CB14F4A2CA890569B1147A594] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382205952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520592437, Process_Duration=2, Process_End_Time=1463520592439, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:52,500 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520592215_0001_1 finished with state: FAILED
2016-05-17 21:29:52,500 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520592215_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:52,500 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520592215_0001_00_000002
2016-05-17 21:29:52,500 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520592215_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:52,501 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592215_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:52 Completed Dag: dag_1463520592215_0001_1
2016-05-17 21:29:52,501 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520592215_0001_1, dagState=FAILED
2016-05-17 21:29:52,501 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520592215_0001_1_post
2016-05-17 21:29:52,501 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:52,501 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:52,501 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520592215_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:52,507 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/ALL_VALUES_REPLACE, applicationId=application_1463520592215_0001
2016-05-17 21:29:52,509 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:52,509 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:52,509 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:52,509 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:52,509 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:52,510 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:52,510 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:52,510 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:52,510 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:52,510 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:52,510 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:52,511 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:52,511 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:52,515 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/02856366506043D193A4276401290753/E2588E98840848BEA43240B4AF79634C/.tez/application_1463520592215_0001
2016-05-17 21:29:52,515 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:52,515 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...atform/ALL_VALUES_REPLACE
2016-05-17 21:29:52,515 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:52,516 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_REPLACE/_temporary
2016-05-17 21:29:52,516 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.319
2016-05-17 21:29:52,516 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_REPLACE
2016-05-17 21:29:52,536 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:52,536 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:52,562 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-05-17 21:29:52,566 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 21:29:52,567 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:52,567 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:52,567 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:52,579 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:52,583 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_SWAP
2016-05-17 21:29:52,605 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:52,605 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:52,624 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:52,624 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:52,624 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_SWAP"]
2016-05-17 21:29:52,624 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:52,625 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:52,625 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:52,625 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...yplatform/ALL_VALUES_SWAP
2016-05-17 21:29:52,625 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:52,637 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:52,641 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:52,642 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:52,642 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:52,651 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/19D442B3D00A4955A0E1D5500D0222F7/A00232CFDF32478984BC0E01B40BCC5F/.tez/application_1463520592641_0001 doesn't exist and is created
2016-05-17 21:29:52,666 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:52,666 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:52,666 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/19D442B3D00A4955A0E1D5500D0222F7/A00232CFDF32478984BC0E01B40BCC5F/.tez/application_1463520592641_0001
2016-05-17 21:29:52,668 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520592641_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:52,678 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:52,678 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:52,679 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:52,679 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:52,679 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/19D442B3D00A4955A0E1D5500D0222F7/A00232CFDF32478984BC0E01B40BCC5F/.tez/application_1463520592641_0001/localmode-log-dir
2016-05-17 21:29:52,679 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/19D442B3D00A4955A0E1D5500D0222F7/A00232CFDF32478984BC0E01B40BCC5F/.tez/application_1463520592641_0001/localmode-log-dir/history.txt.appattempt_1463520592641_0001_000000, maxErrors=10
2016-05-17 21:29:52,680 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:52,680 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520592641_0001_000000, appSubmitTime=1463520592668, launchTime=1463520592668
2016-05-17 21:29:52,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:52,682 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:52,685 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:52,685 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:52,686 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:44354
2016-05-17 21:29:52,688 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:52,690 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520592641_0001_000000, startTime=1463520592690
2016-05-17 21:29:52,690 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:52,690 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:52,766 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:52,766 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:52,766 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/ALL_VALUES_SWAP, applicationId=application_1463520592641_0001, dagName=[19D442B3D00A4955A0E1D5500D0222F7/A00232CFDF32478984BC0E01B40BCC5F] every/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-05-17 21:29:52,783 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [19D442B3D00A4955A0E1D5500D0222F7/A00232CFDF32478984BC0E01B40BCC5F] every/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-05-17 21:29:52,784 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520592641_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/19D442B3D00A4955A0E1D5500D0222F7/A00232CFDF32478984BC0E01B40BCC5F/.tez/application_1463520592641_0001/localmode-log-dir/dag_1463520592641_0001_1.dot
2016-05-17 21:29:52,785 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/19D442B3D00A4955A0E1D5500D0222F7/A00232CFDF32478984BC0E01B40BCC5F/.tez/application_1463520592641_0001/localmode-log-dir/dag_1463520592641_0001_1-tez-dag.pb.txt
2016-05-17 21:29:52,790 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520592641_0001_1
2016-05-17 21:29:52,790 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:52,791 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [19D442B3D00A4955A0E1D5500D0222F7/A00232CFDF32478984BC0E01B40BCC5F] every/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-05-17 21:29:52 Running Dag: dag_1463520592641_0001_1
2016-05-17 21:29:52,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520592641_0001_1, submitTime=1463520592783
2016-05-17 21:29:52,803 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:52,804 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DF92B8D629404575A90269CCAFD1CC06
2016-05-17 21:29:52,804 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:52,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520592641_0001_1, initTime=1463520592803
2016-05-17 21:29:52,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:52,804 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520592641_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:52,805 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:DAG_STARTED]: dagID=dag_1463520592641_0001_1, startTime=1463520592805
2016-05-17 21:29:52,805 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:52,805 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:52,805 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520592641_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:52,805 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B77F87C9AFD946E2B70636A6964BF2E6 : {9FF6B3891BBE40CC83A87AFB6F984937={InputName=9FF6B3891BBE40CC83A87AFB6F984937}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:52,805 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9FF6B3891BBE40CC83A87AFB6F984937, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:52,805 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6]
2016-05-17 21:29:52,805 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6]
2016-05-17 21:29:52,806 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6]
2016-05-17 21:29:52,806 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6]. Starting root input initializers: 1
2016-05-17 21:29:52,806 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:52,806 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9FF6B3891BBE40CC83A87AFB6F984937 on vertex vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6]
2016-05-17 21:29:52,806 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06]
2016-05-17 21:29:52,808 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:52,808 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/ALL_VALUES_SWAP, applicationId=application_1463520592641_0001, dagName=[19D442B3D00A4955A0E1D5500D0222F7/A00232CFDF32478984BC0E01B40BCC5F] every/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-05-17 21:29:52,808 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9FF6B3891BBE40CC83A87AFB6F984937 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:52,808 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:52,809 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:52,809 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:52,809 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06]
2016-05-17 21:29:52,810 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06]
2016-05-17 21:29:52,810 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06]
2016-05-17 21:29:52,810 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=94E50FBF4145479AA039FD5798F3F420, vertexId=vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:52,810 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=94E50FBF4145479AA039FD5798F3F420, vertexId=vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06]
2016-05-17 21:29:52,811 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:52,811 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9FF6B3891BBE40CC83A87AFB6F984937 on vertex vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6]
2016-05-17 21:29:52,812 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=94E50FBF4145479AA039FD5798F3F420, vertexName=DF92B8D629404575A90269CCAFD1CC06, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:52,812 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=94E50FBF4145479AA039FD5798F3F420, vertexId=vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06]
2016-05-17 21:29:52,813 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:52,813 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DF92B8D629404575A90269CCAFD1CC06, vertexId=vertex_1463520592641_0001_1_01, initRequestedTime=1463520592806, initedTime=1463520592813, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:52,813 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:52,814 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:52,814 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9FF6B3891BBE40CC83A87AFB6F984937=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:52,814 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6] parallelism set to 1
2016-05-17 21:29:52,814 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:52,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520592641_0001_1_00, eventCount=1
2016-05-17 21:29:52,815 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:52,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B77F87C9AFD946E2B70636A6964BF2E6, vertexId=vertex_1463520592641_0001_1_00, initRequestedTime=1463520592805, initedTime=1463520592815, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:52,815 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6] with distanceFromRoot: 0
2016-05-17 21:29:52,816 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:52,816 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B77F87C9AFD946E2B70636A6964BF2E6
2016-05-17 21:29:52,816 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6]
2016-05-17 21:29:52,816 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520592641_0001_1_00, startRequestedTime=1463520592816, startedTime=1463520592816
2016-05-17 21:29:52,816 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:52,817 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:52,817 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:52,818 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:TASK_STARTED]: vertexName=B77F87C9AFD946E2B70636A6964BF2E6, taskId=task_1463520592641_0001_1_00_000000, scheduledTime=1463520592817, launchTime=1463520592817
2016-05-17 21:29:52,818 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520592641_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:52,818 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520592641_0001_1_00 for vertex: vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06] numStartedSources: 1 numSources: 1
2016-05-17 21:29:52,818 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06] with distanceFromRoot: 1
2016-05-17 21:29:52,818 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6]
2016-05-17 21:29:52,819 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B77F87C9AFD946E2B70636A6964BF2E6 in vertex: DF92B8D629404575A90269CCAFD1CC06
2016-05-17 21:29:52,819 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DF92B8D629404575A90269CCAFD1CC06 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:52,819 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:52,820 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520592641_0001_1_01, startRequestedTime=1463520592819, startedTime=1463520592819
2016-05-17 21:29:52,820 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:52,820 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [19D442B3D00A4955A0E1D5500D0222F7/A00232CFDF32478984BC0E01B40BCC5F] every/(1/1) ...yplatform/ALL_VALUES_SWAP, VertexName: B77F87C9AFD946E2B70636A6964BF2E6, VertexParallelism: 1, TaskAttemptID:attempt_1463520592641_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9FF6B3891BBE40CC83A87AFB6F984937, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DF92B8D629404575A90269CCAFD1CC06, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:52,820 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520592641_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:52,820 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:52,821 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:52,821 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:52,822 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:52,822 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520592641_0001_00_000001
2016-05-17 21:29:52,822 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592641_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:52,823 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520592641_0001_00_000001
2016-05-17 21:29:52,824 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:52,825 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592641_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:52,825 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:52,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520592641_0001_00_000001, launchTime=1463520592825
2016-05-17 21:29:52,825 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520592641_0001_00_000001
2016-05-17 21:29:52,826 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520592641_0001, with dag id: dag_1463520592641_0001_1
2016-05-17 21:29:52,826 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520592641_0001_1_00_000000_0] to container: [container_1463520592641_0001_00_000001]
2016-05-17 21:29:52,826 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592641_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:52,826 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520592641_0001_00_000001 given task: attempt_1463520592641_0001_1_00_000000_0
2016-05-17 21:29:52,826 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520592641_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520592641_0001_1_00_000000_0
2016-05-17 21:29:52,827 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520592641_0001_1_00_000000_0] started. Is using containerId: [container_1463520592641_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:52 Starting to run new task attempt: attempt_1463520592641_0001_1_00_000000_0
2016-05-17 21:29:52,827 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520592641_0001_1_00_000000_0
2016-05-17 21:29:52,827 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:52,827 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:52,827 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:52,827 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:52,827 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:52,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B77F87C9AFD946E2B70636A6964BF2E6, taskAttemptId=attempt_1463520592641_0001_1_00_000000_0, startTime=1463520592826, containerId=container_1463520592641_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520592641_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:52,827 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:52,828 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520592641_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:52,828 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520592641_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:52,827 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:52,828 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [19D442B3D00A4955A0E1D5500D0222F7/A00232CFDF32478984BC0E01B40BCC5F] every/(1/1) ...yplatform/ALL_VALUES_SWAP, VertexName: B77F87C9AFD946E2B70636A6964BF2E6, VertexParallelism: 1, TaskAttemptID:attempt_1463520592641_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9FF6B3891BBE40CC83A87AFB6F984937, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DF92B8D629404575A90269CCAFD1CC06, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:52,828 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:52,829 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520592641_0001_1_00_000000_0
2016-05-17 21:29:52,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:52,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:52,830 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520592641_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:52,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9FF6B3891BBE40CC83A87AFB6F984937, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:52,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DF92B8D629404575A90269CCAFD1CC06, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:52,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:52,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:52,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DF92B8D629404575A90269CCAFD1CC06
2016-05-17 21:29:52,832 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:52,832 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DF92B8D629404575A90269CCAFD1CC06
2016-05-17 21:29:52,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DF92B8D629404575A90269CCAFD1CC06
2016-05-17 21:29:52,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:52,833 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9FF6B3891BBE40CC83A87AFB6F984937
2016-05-17 21:29:52,833 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9FF6B3891BBE40CC83A87AFB6F984937
2016-05-17 21:29:52,833 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:52,834 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:52,834 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9FF6B3891BBE40CC83A87AFB6F984937
2016-05-17 21:29:52,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9FF6B3891BBE40CC83A87AFB6F984937
2016-05-17 21:29:52,837 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B77F87C9AFD946E2B70636A6964BF2E6, ordinal: 0
2016-05-17 21:29:52,837 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B77F87C9AFD946E2B70636A6964BF2E6, mem on start (mb), free: 101, total: 364, max: 672
2016-05-17 21:29:52,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:52,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:52,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:52,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:52,838 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:52,838 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:52,838 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:52,838 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:52,839 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:52,839 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:52,839 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DF92B8D629404575A90269CCAFD1CC06, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:52,839 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9FF6B3891BBE40CC83A87AFB6F984937, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:52,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:52,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9FF6B3891BBE40CC83A87AFB6F984937 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:52,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:52,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:52,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9FF6B3891BBE40CC83A87AFB6F984937
2016-05-17 21:29:52,840 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:52,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9FF6B3891BBE40CC83A87AFB6F984937
2016-05-17 21:29:52,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:52,840 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520592641_0001_1_00_000000_0
2016-05-17 21:29:52,841 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:52,842 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2233f372
2016-05-17 21:29:52,843 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2233f372
2016-05-17 21:29:52,843 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:52,843 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:52,843 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:52,844 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 9FF6B3891BBE40CC83A87AFB6F984937
2016-05-17 21:29:52,844 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 1059C4B6024E4FACB2BF7ACB189275A7
2016-05-17 21:29:52,845 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 1059C4B6024E4FACB2BF7ACB189275A7
2016-05-17 21:29:52,846 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:52,848 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:52,849 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@6b4ca849; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@75bc87a7; comparator=cascading.tuple.hadoop.util.TupleComparator@2c6e2154; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:52,849 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:52,849 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:52,849 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:52,872 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:52,872 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:52,872 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:52,872 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:52,873 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:52,873 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:52,873 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:52,873 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B77F87C9AFD946E2B70636A6964BF2E6, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:52,876 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520592641_0001_1_00_000000_0
2016-05-17 21:29:52,876 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B77F87C9AFD946E2B70636A6964BF2E6, mem on close (mb), free: 184, total: 365, max: 672
2016-05-17 21:29:52,876 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:52,876 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:52,876 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:52,876 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:52,880 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:52 Completed running task attempt: attempt_1463520592641_0001_1_00_000000_0
2016-05-17 21:29:52,882 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520592641_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:52,883 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520592874, Process_Duration=2, Process_End_Time=1463520592876, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:52,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520592641_0001_1_00, eventCount=1
2016-05-17 21:29:52,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B77F87C9AFD946E2B70636A6964BF2E6, taskAttemptId=attempt_1463520592641_0001_1_00_000000_0, startTime=1463520592826, finishTime=1463520592883, timeTaken=57, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520592874, Process_Duration=2, Process_End_Time=1463520592876, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:52,883 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520592641_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:52,884 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:52,884 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520592641_0001_1_00_000000_0
2016-05-17 21:29:52,884 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:52,884 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:52,884 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:52,884 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520592641_0001_00_000001
2016-05-17 21:29:52,884 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:TASK_FINISHED]: vertexName=B77F87C9AFD946E2B70636A6964BF2E6, taskId=task_1463520592641_0001_1_00_000000, startTime=1463520592826, finishTime=1463520592884, timeTaken=58, status=SUCCEEDED, successfulAttemptID=attempt_1463520592641_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520592874, Process_Duration=2, Process_End_Time=1463520592876, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:52,884 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:52,884 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520592641_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:52,885 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6] : 1
2016-05-17 21:29:52,885 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6]
2016-05-17 21:29:52,885 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:52,885 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:VERTEX_FINISHED]: vertexName=B77F87C9AFD946E2B70636A6964BF2E6, vertexId=vertex_1463520592641_0001_1_00, initRequestedTime=1463520592805, initedTime=1463520592815, startRequestedTime=1463520592816, startedTime=1463520592816, finishTime=1463520592885, timeTaken=69, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520592874, Process_Duration=2, Process_End_Time=1463520592876, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520592826, firstTasksToStart=[ task_1463520592641_0001_1_00_000000 ], lastTaskFinishTime=1463520592883, lastTasksToFinish=[ task_1463520592641_0001_1_00_000000 ], minTaskDuration=57, maxTaskDuration=57, avgTaskDuration=57.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520592641_0001_1_00_000000 ], longestDurationTasks=[ task_1463520592641_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:52,885 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:52,885 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592641_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:52,886 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520592641_0001_00_000001, stoppedTime=1463520592886, exitStatus=0
2016-05-17 21:29:52,886 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:52,886 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592641_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:52,886 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06] attempt: attempt_1463520592641_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:52,886 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DF92B8D629404575A90269CCAFD1CC06
2016-05-17 21:29:52,887 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06]
2016-05-17 21:29:52,887 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:52,887 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520592641_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:52,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:TASK_STARTED]: vertexName=DF92B8D629404575A90269CCAFD1CC06, taskId=task_1463520592641_0001_1_01_000000, scheduledTime=1463520592887, launchTime=1463520592887
2016-05-17 21:29:52,887 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:52,887 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520592641_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:52,888 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592641_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:52,888 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06]
2016-05-17 21:29:52,888 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [19D442B3D00A4955A0E1D5500D0222F7/A00232CFDF32478984BC0E01B40BCC5F] every/(1/1) ...yplatform/ALL_VALUES_SWAP, VertexName: DF92B8D629404575A90269CCAFD1CC06, VertexParallelism: 1, TaskAttemptID:attempt_1463520592641_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B77F87C9AFD946E2B70636A6964BF2E6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=94E50FBF4145479AA039FD5798F3F420, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:52,888 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520592641_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:52,889 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:52,890 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520592641_0001_00_000002
2016-05-17 21:29:52,890 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592641_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:52,892 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520592641_0001_00_000002
2016-05-17 21:29:52,892 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:52,893 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592641_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:52,893 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:52,893 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520592641_0001_00_000002, launchTime=1463520592893
2016-05-17 21:29:52,893 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520592641_0001_00_000002
2016-05-17 21:29:52,894 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520592641_0001_1_01_000000_0] to container: [container_1463520592641_0001_00_000002]
2016-05-17 21:29:52,894 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592641_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:52,894 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520592641_0001_00_000002 given task: attempt_1463520592641_0001_1_01_000000_0
2016-05-17 21:29:52,894 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520592641_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520592641_0001_1_01_000000_0
2016-05-17 21:29:52,894 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520592641_0001_1_01_000000_0] started. Is using containerId: [container_1463520592641_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:52,894 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520592641_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:52,895 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DF92B8D629404575A90269CCAFD1CC06, taskAttemptId=attempt_1463520592641_0001_1_01_000000_0, startTime=1463520592894, containerId=container_1463520592641_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520592641_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:52 Starting to run new task attempt: attempt_1463520592641_0001_1_01_000000_0
2016-05-17 21:29:52,895 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520592641_0001_1_01_000000_0
2016-05-17 21:29:52,895 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520592641_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:52,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:52,895 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520592641_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:52,895 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520592641_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:52,895 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:52,895 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520592641_0001_00_000001, Exiting
2016-05-17 21:29:52,896 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520592641_0001_00_000001
2016-05-17 21:29:52,896 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520592641_0001_00_000001
2016-05-17 21:29:52,896 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:52,896 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:52,896 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:52,896 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:52,896 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:52,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [19D442B3D00A4955A0E1D5500D0222F7/A00232CFDF32478984BC0E01B40BCC5F] every/(1/1) ...yplatform/ALL_VALUES_SWAP, VertexName: DF92B8D629404575A90269CCAFD1CC06, VertexParallelism: 1, TaskAttemptID:attempt_1463520592641_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B77F87C9AFD946E2B70636A6964BF2E6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=94E50FBF4145479AA039FD5798F3F420, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:52,896 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:52,897 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520592641_0001_1_01_000000_0
2016-05-17 21:29:52,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:52,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:52,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:52,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B77F87C9AFD946E2B70636A6964BF2E6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:52,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:52,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B77F87C9AFD946E2B70636A6964BF2E6
2016-05-17 21:29:52,899 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520592641_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:52,899 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520592641_0001_00_000001 completed successfully
2016-05-17 21:29:52,899 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592641_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:52,899 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:52,899 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:52,899 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B77F87C9AFD946E2B70636A6964BF2E6
2016-05-17 21:29:52,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B77F87C9AFD946E2B70636A6964BF2E6
2016-05-17 21:29:52,900 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520592641_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:52,900 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:52,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=94E50FBF4145479AA039FD5798F3F420, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:52,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:52,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 94E50FBF4145479AA039FD5798F3F420
2016-05-17 21:29:52,906 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:52,907 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 94E50FBF4145479AA039FD5798F3F420
2016-05-17 21:29:52,904 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DF92B8D629404575A90269CCAFD1CC06, ordinal: 1
2016-05-17 21:29:52,908 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DF92B8D629404575A90269CCAFD1CC06, mem on start (mb), free: 178, total: 365, max: 672
2016-05-17 21:29:52,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:52,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:52,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:52,914 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:52,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 94E50FBF4145479AA039FD5798F3F420
2016-05-17 21:29:52,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:52,914 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:52,915 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:52,915 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:52,915 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:52,915 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:52,916 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:52,916 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B77F87C9AFD946E2B70636A6964BF2E6, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:52,916 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 94E50FBF4145479AA039FD5798F3F420, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:52,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:52,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B77F87C9AFD946E2B70636A6964BF2E6 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:52,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B77F87C9AFD946E2B70636A6964BF2E6
2016-05-17 21:29:52,917 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:52,917 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:52,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:52,917 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B77F87C9AFD946E2B70636A6964BF2E6 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:52,918 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:52,918 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:52,918 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:52,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:52,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B77F87C9AFD946E2B70636A6964BF2E6
2016-05-17 21:29:52,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:52,919 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520592641_0001_1_01_000000_0
2016-05-17 21:29:52,920 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520592641_0001_1_00_000000_0_10315, runDuration: 49779, ]
2016-05-17 21:29:52,921 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:52,921 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 1059C4B6024E4FACB2BF7ACB189275A7
2016-05-17 21:29:52,921 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_SWAP"], id: 94E50FBF4145479AA039FD5798F3F420
2016-05-17 21:29:52,921 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520592641_0001_1_00_000000_0_10315] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520592921, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:52,922 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B77F87C9AFD946E2B70636A6964BF2E6
2016-05-17 21:29:52,922 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_SWAP"]
2016-05-17 21:29:52,922 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:52,922 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 1059C4B6024E4FACB2BF7ACB189275A7, for 1 inputs
2016-05-17 21:29:52,922 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B77F87C9AFD946E2B70636A6964BF2E6] #53 in 1ms
2016-05-17 21:29:52,923 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B77F87C9AFD946E2B70636A6964BF2E6] #53, status:false, isInterrupted:false
2016-05-17 21:29:52,923 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:52,924 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:52,924 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:52,924 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:52,924 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:52,925 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:52,925 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B77F87C9AFD946E2B70636A6964BF2E6
2016-05-17 21:29:52,925 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:52,925 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DF92B8D629404575A90269CCAFD1CC06, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:52,926 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:52,927 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@1b9b760; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@47b60082
2016-05-17 21:29:52,928 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:52,928 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:52,928 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:52,928 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:52,929 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520592641_0001_1_01_000000_0
2016-05-17 21:29:52,929 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520592641_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:52,929 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520592641_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:52,932 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:52,932 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DF92B8D629404575A90269CCAFD1CC06, taskAttemptId=attempt_1463520592641_0001_1_01_000000_0, startTime=1463520592894, finishTime=1463520592932, timeTaken=38, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520592925, Process_Duration=4, Process_End_Time=1463520592929, Read_Duration=0, Tuples_Read=1
2016-05-17 21:29:52,932 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520592641_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:52,932 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520592641_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:52,932 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:TASK_FINISHED]: vertexName=DF92B8D629404575A90269CCAFD1CC06, taskId=task_1463520592641_0001_1_01_000000, startTime=1463520592894, finishTime=1463520592932, timeTaken=38, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:52,932 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:52,932 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520592641_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:52,933 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06] : 1
2016-05-17 21:29:52,933 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06] because task failed: task_1463520592641_0001_1_01_000000
2016-05-17 21:29:52,933 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:52,933 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:52,933 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:52,933 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06]
2016-05-17 21:29:52,933 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=94E50FBF4145479AA039FD5798F3F420, vertexId=vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06]
2016-05-17 21:29:52,934 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:52,934 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:VERTEX_FINISHED]: vertexName=DF92B8D629404575A90269CCAFD1CC06, vertexId=vertex_1463520592641_0001_1_01, initRequestedTime=1463520592806, initedTime=1463520592813, startRequestedTime=1463520592819, startedTime=1463520592819, finishTime=1463520592934, timeTaken=115, status=FAILED, diagnostics=Task failed, taskId=task_1463520592641_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520592894, firstTasksToStart=[ task_1463520592641_0001_1_01_000000 ], lastTaskFinishTime=1463520592932, lastTasksToFinish=[ task_1463520592641_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:52,935 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:52,935 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520592925, Process_Duration=4, Process_End_Time=1463520592929, Read_Duration=0, Tuples_Read=1]]
2016-05-17 21:29:52,935 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:52,935 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DF92B8D629404575A90269CCAFD1CC06, mem on close (mb), free: 174, total: 365, max: 672
2016-05-17 21:29:52,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=DF92B8D629404575A90269CCAFD1CC06, index=1
2016-05-17 21:29:52,937 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B77F87C9AFD946E2B70636A6964BF2E6
2016-05-17 21:29:52,937 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:52,935 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520592641_0001_00_000002, stoppedTime=1463520592935, exitStatus=0
2016-05-17 21:29:52,937 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592641_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:52,937 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520592641_0001_1_01_000000_0 failed: true container: container_1463520592641_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:52,937 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:52,937 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:52,937 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520592641_0001_1
2016-05-17 21:29:52,937 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 94E50FBF4145479AA039FD5798F3F420 for vertex: vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06]
2016-05-17 21:29:52,938 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520592641_0001_1_00 [B77F87C9AFD946E2B70636A6964BF2E6]
2016-05-17 21:29:52,938 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:52,938 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:52,938 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:52,938 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:52,938 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:52,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=DF92B8D629404575A90269CCAFD1CC06, sourceVertex=B77F87C9AFD946E2B70636A6964BF2E6
2016-05-17 21:29:52,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=DF92B8D629404575A90269CCAFD1CC06, sourceVertex=94E50FBF4145479AA039FD5798F3F420
2016-05-17 21:29:52,939 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520592641_0001_1, queueSize=0
2016-05-17 21:29:52,939 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520592641_0001_00_000002
2016-05-17 21:29:52,939 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520592641_0001_00_000002
2016-05-17 21:29:52,939 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520592641_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520592641_0001_1, startTime=1463520592805, finishTime=1463520592938, timeTaken=133, status=FAILED, diagnostics=Vertex failed, vertexName=DF92B8D629404575A90269CCAFD1CC06, vertexId=vertex_1463520592641_0001_1_01, diagnostics=[Task failed, taskId=task_1463520592641_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:VALUES]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520592641_0001_1_01 [DF92B8D629404575A90269CCAFD1CC06] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=1249, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=382730240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520592874, Process_Duration=2, Process_End_Time=1463520592876, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:52,939 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:52,939 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520592641_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:52,939 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520592641_0001_00_000002 completed but with errors
2016-05-17 21:29:52,940 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520592641_0001_1 finished with state: FAILED
2016-05-17 21:29:52,940 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520592641_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:52,940 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520592641_0001_00_000002
2016-05-17 21:29:52,940 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520592641_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:52,940 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520592641_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:52 Completed Dag: dag_1463520592641_0001_1
2016-05-17 21:29:52,940 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520592641_0001_1, dagState=FAILED
2016-05-17 21:29:52,940 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520592641_0001_1_post
2016-05-17 21:29:52,940 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:52,940 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:52,941 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520592641_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:52,943 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/ALL_VALUES_SWAP, applicationId=application_1463520592641_0001
2016-05-17 21:29:52,945 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:52,945 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:52,945 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:52,945 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:52,945 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:52,946 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:52,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:52,946 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:52,946 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:52,946 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:52,947 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:52,947 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:52,947 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:52,950 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/19D442B3D00A4955A0E1D5500D0222F7/A00232CFDF32478984BC0E01B40BCC5F/.tez/application_1463520592641_0001
2016-05-17 21:29:52,951 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:52,951 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...yplatform/ALL_VALUES_SWAP
2016-05-17 21:29:52,952 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:52,952 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_SWAP/_temporary
2016-05-17 21:29:52,952 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.328
2016-05-17 21:29:52,952 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_SWAP
2016-05-17 21:29:52,982 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:52,982 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:53,013 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-05-17 21:29:53,023 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-05-17 21:29:53,024 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:53,024 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:53,024 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:53,043 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:53,047 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_-1
2016-05-17 21:29:53,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:53,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:53,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_-1"]
2016-05-17 21:29:53,087 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:53,088 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:53,088 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:53,089 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...emblyplatform/ALL_ARGS_-1
2016-05-17 21:29:53,089 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:53,100 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:53,104 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:53,104 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:53,104 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:53,113 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8089A5942E8341AE9AB81BEFC77D4422/D44F4D90A5FB4CC99C237E1C8DBC0C58/.tez/application_1463520593104_0001 doesn't exist and is created
2016-05-17 21:29:53,127 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:53,127 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:53,127 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8089A5942E8341AE9AB81BEFC77D4422/D44F4D90A5FB4CC99C237E1C8DBC0C58/.tez/application_1463520593104_0001
2016-05-17 21:29:53,129 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520593104_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:53,136 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:53,136 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:53,138 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:53,138 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:53,139 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:53,139 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:53,139 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8089A5942E8341AE9AB81BEFC77D4422/D44F4D90A5FB4CC99C237E1C8DBC0C58/.tez/application_1463520593104_0001/localmode-log-dir
2016-05-17 21:29:53,139 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8089A5942E8341AE9AB81BEFC77D4422/D44F4D90A5FB4CC99C237E1C8DBC0C58/.tez/application_1463520593104_0001/localmode-log-dir/history.txt.appattempt_1463520593104_0001_000000, maxErrors=10
2016-05-17 21:29:53,139 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:53,139 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520593104_0001_000000, appSubmitTime=1463520593128, launchTime=1463520593128
2016-05-17 21:29:53,141 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:53,142 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:53,143 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:53,144 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:53,145 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:44622
2016-05-17 21:29:53,146 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:53,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520593104_0001_000000, startTime=1463520593147
2016-05-17 21:29:53,147 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:53,147 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:53,227 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:53,227 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:53,227 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...emblyplatform/ALL_ARGS_-1, applicationId=application_1463520593104_0001, dagName=[8089A5942E8341AE9AB81BEFC77D4422/D44F4D90A5FB4CC99C237E1C8DBC0C58] every/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-05-17 21:29:53,241 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8089A5942E8341AE9AB81BEFC77D4422/D44F4D90A5FB4CC99C237E1C8DBC0C58] every/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-05-17 21:29:53,242 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520593104_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8089A5942E8341AE9AB81BEFC77D4422/D44F4D90A5FB4CC99C237E1C8DBC0C58/.tez/application_1463520593104_0001/localmode-log-dir/dag_1463520593104_0001_1.dot
2016-05-17 21:29:53,242 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8089A5942E8341AE9AB81BEFC77D4422/D44F4D90A5FB4CC99C237E1C8DBC0C58/.tez/application_1463520593104_0001/localmode-log-dir/dag_1463520593104_0001_1-tez-dag.pb.txt
2016-05-17 21:29:53,248 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520593104_0001_1
2016-05-17 21:29:53,248 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:53,248 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8089A5942E8341AE9AB81BEFC77D4422/D44F4D90A5FB4CC99C237E1C8DBC0C58] every/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-05-17 21:29:53 Running Dag: dag_1463520593104_0001_1
2016-05-17 21:29:53,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520593104_0001_1, submitTime=1463520593241
2016-05-17 21:29:53,259 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:53,260 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B3AD3CC7DA4947F4AEE5F954412D8792
2016-05-17 21:29:53,260 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:53,260 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:53,260 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520593104_0001_1, initTime=1463520593259
2016-05-17 21:29:53,260 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520593104_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:53,261 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:DAG_STARTED]: dagID=dag_1463520593104_0001_1, startTime=1463520593261
2016-05-17 21:29:53,261 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:53,261 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...emblyplatform/ALL_ARGS_-1, applicationId=application_1463520593104_0001, dagName=[8089A5942E8341AE9AB81BEFC77D4422/D44F4D90A5FB4CC99C237E1C8DBC0C58] every/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-05-17 21:29:53,261 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:53,261 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520593104_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:53,262 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B1666ED4517A412780E5DAC1FCB6D838 : {3ECD84D0493A4742B9A3B6EF6E375E97={InputName=3ECD84D0493A4742B9A3B6EF6E375E97}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:53,262 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 3ECD84D0493A4742B9A3B6EF6E375E97, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:53,262 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838]
2016-05-17 21:29:53,263 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838]
2016-05-17 21:29:53,263 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838]
2016-05-17 21:29:53,263 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838]. Starting root input initializers: 1
2016-05-17 21:29:53,263 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:53,263 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792]
2016-05-17 21:29:53,264 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 3ECD84D0493A4742B9A3B6EF6E375E97 on vertex vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838]
2016-05-17 21:29:53,266 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:53,266 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:53,266 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792]
2016-05-17 21:29:53,267 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792]
2016-05-17 21:29:53,267 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792]
2016-05-17 21:29:53,267 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B4BEBB14E0FD42E389F60AA86019AA0A, vertexId=vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:53,267 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B4BEBB14E0FD42E389F60AA86019AA0A, vertexId=vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792]
2016-05-17 21:29:53,267 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 3ECD84D0493A4742B9A3B6EF6E375E97 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:53,268 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:53,268 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:53,269 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B4BEBB14E0FD42E389F60AA86019AA0A, vertexName=B3AD3CC7DA4947F4AEE5F954412D8792, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:53,269 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B4BEBB14E0FD42E389F60AA86019AA0A, vertexId=vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792]
2016-05-17 21:29:53,270 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:53,270 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B3AD3CC7DA4947F4AEE5F954412D8792, vertexId=vertex_1463520593104_0001_1_01, initRequestedTime=1463520593263, initedTime=1463520593270, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:53,270 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:53,271 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:53,271 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 3ECD84D0493A4742B9A3B6EF6E375E97 on vertex vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838]
2016-05-17 21:29:53,272 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:53,272 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {3ECD84D0493A4742B9A3B6EF6E375E97=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:53,272 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838] parallelism set to 1
2016-05-17 21:29:53,272 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:53,272 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520593104_0001_1_00, eventCount=1
2016-05-17 21:29:53,273 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:53,273 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B1666ED4517A412780E5DAC1FCB6D838, vertexId=vertex_1463520593104_0001_1_00, initRequestedTime=1463520593262, initedTime=1463520593272, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:53,273 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838] with distanceFromRoot: 0
2016-05-17 21:29:53,273 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:53,273 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B1666ED4517A412780E5DAC1FCB6D838
2016-05-17 21:29:53,273 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838]
2016-05-17 21:29:53,274 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:53,274 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520593104_0001_1_00, startRequestedTime=1463520593273, startedTime=1463520593273
2016-05-17 21:29:53,274 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:53,274 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:53,274 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:TASK_STARTED]: vertexName=B1666ED4517A412780E5DAC1FCB6D838, taskId=task_1463520593104_0001_1_00_000000, scheduledTime=1463520593274, launchTime=1463520593274
2016-05-17 21:29:53,274 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520593104_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:53,275 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520593104_0001_1_00 for vertex: vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792] numStartedSources: 1 numSources: 1
2016-05-17 21:29:53,275 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792] with distanceFromRoot: 1
2016-05-17 21:29:53,275 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838]
2016-05-17 21:29:53,275 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B1666ED4517A412780E5DAC1FCB6D838 in vertex: B3AD3CC7DA4947F4AEE5F954412D8792
2016-05-17 21:29:53,275 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B3AD3CC7DA4947F4AEE5F954412D8792 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:53,275 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:53,276 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520593104_0001_1_01, startRequestedTime=1463520593275, startedTime=1463520593275
2016-05-17 21:29:53,276 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:53,276 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8089A5942E8341AE9AB81BEFC77D4422/D44F4D90A5FB4CC99C237E1C8DBC0C58] every/(1/1) ...emblyplatform/ALL_ARGS_-1, VertexName: B1666ED4517A412780E5DAC1FCB6D838, VertexParallelism: 1, TaskAttemptID:attempt_1463520593104_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3ECD84D0493A4742B9A3B6EF6E375E97, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B3AD3CC7DA4947F4AEE5F954412D8792, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:53,276 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520593104_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:53,277 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:53,277 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:53,278 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:53,278 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:53,278 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520593104_0001_00_000001
2016-05-17 21:29:53,278 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593104_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:53,280 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520593104_0001_00_000001
2016-05-17 21:29:53,280 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:53,281 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593104_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:53,281 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:53,281 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520593104_0001_00_000001, launchTime=1463520593281
2016-05-17 21:29:53,281 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520593104_0001, with dag id: dag_1463520593104_0001_1
2016-05-17 21:29:53,282 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520593104_0001_00_000001
2016-05-17 21:29:53,282 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520593104_0001_1_00_000000_0] to container: [container_1463520593104_0001_00_000001]
2016-05-17 21:29:53,282 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593104_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:53,283 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520593104_0001_1_00_000000_0] started. Is using containerId: [container_1463520593104_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:53,283 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B1666ED4517A412780E5DAC1FCB6D838, taskAttemptId=attempt_1463520593104_0001_1_00_000000_0, startTime=1463520593283, containerId=container_1463520593104_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520593104_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:53,283 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520593104_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:53,283 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520593104_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:53,284 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:53,284 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520593104_0001_00_000001 given task: attempt_1463520593104_0001_1_00_000000_0
2016-05-17 21:29:53,284 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520593104_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520593104_0001_1_00_000000_0
2016-05-17 21:29:53 Starting to run new task attempt: attempt_1463520593104_0001_1_00_000000_0
2016-05-17 21:29:53,285 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520593104_0001_1_00_000000_0
2016-05-17 21:29:53,285 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:53,285 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:53,285 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:53,285 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:53,285 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:53,285 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:53,285 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8089A5942E8341AE9AB81BEFC77D4422/D44F4D90A5FB4CC99C237E1C8DBC0C58] every/(1/1) ...emblyplatform/ALL_ARGS_-1, VertexName: B1666ED4517A412780E5DAC1FCB6D838, VertexParallelism: 1, TaskAttemptID:attempt_1463520593104_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=3ECD84D0493A4742B9A3B6EF6E375E97, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B3AD3CC7DA4947F4AEE5F954412D8792, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:53,286 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:53,287 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520593104_0001_1_00_000000_0
2016-05-17 21:29:53,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:53,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:53,289 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520593104_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:53,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=3ECD84D0493A4742B9A3B6EF6E375E97, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:53,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:53,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 3ECD84D0493A4742B9A3B6EF6E375E97
2016-05-17 21:29:53,290 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 3ECD84D0493A4742B9A3B6EF6E375E97
2016-05-17 21:29:53,291 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:53,292 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 3ECD84D0493A4742B9A3B6EF6E375E97
2016-05-17 21:29:53,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 3ECD84D0493A4742B9A3B6EF6E375E97
2016-05-17 21:29:53,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:53,294 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:53,296 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B1666ED4517A412780E5DAC1FCB6D838, ordinal: 0
2016-05-17 21:29:53,296 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1666ED4517A412780E5DAC1FCB6D838, mem on start (mb), free: 101, total: 365, max: 672
2016-05-17 21:29:53,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:53,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:53,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:53,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B3AD3CC7DA4947F4AEE5F954412D8792, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:53,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:53,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B3AD3CC7DA4947F4AEE5F954412D8792
2016-05-17 21:29:53,299 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:53,299 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B3AD3CC7DA4947F4AEE5F954412D8792
2016-05-17 21:29:53,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B3AD3CC7DA4947F4AEE5F954412D8792
2016-05-17 21:29:53,300 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:53,300 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:53,300 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:53,300 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:53,300 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:53,300 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:53,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:53,301 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 3ECD84D0493A4742B9A3B6EF6E375E97, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:53,301 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B3AD3CC7DA4947F4AEE5F954412D8792, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:53,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:53,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 3ECD84D0493A4742B9A3B6EF6E375E97 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:53,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:53,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:53,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 3ECD84D0493A4742B9A3B6EF6E375E97
2016-05-17 21:29:53,301 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:53,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 3ECD84D0493A4742B9A3B6EF6E375E97
2016-05-17 21:29:53,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:53,302 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520593104_0001_1_00_000000_0
2016-05-17 21:29:53,303 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:53,303 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@321cf419
2016-05-17 21:29:53,304 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@321cf419
2016-05-17 21:29:53,304 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:53,304 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:53,305 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:53,305 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 3ECD84D0493A4742B9A3B6EF6E375E97
2016-05-17 21:29:53,305 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: FE864B2C80CB4907962F27D305625047
2016-05-17 21:29:53,306 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] FE864B2C80CB4907962F27D305625047
2016-05-17 21:29:53,306 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:53,308 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:53,308 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@34ac5b97; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@1fb13185; comparator=cascading.tuple.hadoop.util.TupleComparator@2a2a2304; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:53,308 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:53,309 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:53,309 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:53,333 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:53,334 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:53,334 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:53,334 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:53,334 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:53,335 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:53,335 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:53,336 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B1666ED4517A412780E5DAC1FCB6D838, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:53,338 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520593104_0001_1_00_000000_0
2016-05-17 21:29:53,339 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1666ED4517A412780E5DAC1FCB6D838, mem on close (mb), free: 184, total: 366, max: 672
2016-05-17 21:29:53,339 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:53,339 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:53,339 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:53,339 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:53,343 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:53 Completed running task attempt: attempt_1463520593104_0001_1_00_000000_0
2016-05-17 21:29:53,346 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520593104_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:53,346 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520593336, Process_Duration=2, Process_End_Time=1463520593338, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:53,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520593104_0001_1_00, eventCount=1
2016-05-17 21:29:53,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:53,347 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B1666ED4517A412780E5DAC1FCB6D838, taskAttemptId=attempt_1463520593104_0001_1_00_000000_0, startTime=1463520593283, finishTime=1463520593346, timeTaken=63, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520593336, Process_Duration=2, Process_End_Time=1463520593338, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:53,347 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520593104_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:53,347 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:53,347 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520593104_0001_1_00_000000_0
2016-05-17 21:29:53,347 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:53,347 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:53,347 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:53,347 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:TASK_FINISHED]: vertexName=B1666ED4517A412780E5DAC1FCB6D838, taskId=task_1463520593104_0001_1_00_000000, startTime=1463520593283, finishTime=1463520593347, timeTaken=64, status=SUCCEEDED, successfulAttemptID=attempt_1463520593104_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520593336, Process_Duration=2, Process_End_Time=1463520593338, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:53,347 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520593104_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:53,348 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838] : 1
2016-05-17 21:29:53,348 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838]
2016-05-17 21:29:53,348 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520593104_0001_00_000001
2016-05-17 21:29:53,348 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:VERTEX_FINISHED]: vertexName=B1666ED4517A412780E5DAC1FCB6D838, vertexId=vertex_1463520593104_0001_1_00, initRequestedTime=1463520593262, initedTime=1463520593272, startRequestedTime=1463520593273, startedTime=1463520593273, finishTime=1463520593348, timeTaken=75, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520593336, Process_Duration=2, Process_End_Time=1463520593338, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520593283, firstTasksToStart=[ task_1463520593104_0001_1_00_000000 ], lastTaskFinishTime=1463520593346, lastTasksToFinish=[ task_1463520593104_0001_1_00_000000 ], minTaskDuration=63, maxTaskDuration=63, avgTaskDuration=63.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520593104_0001_1_00_000000 ], longestDurationTasks=[ task_1463520593104_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:53,348 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:53,348 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593104_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:53,349 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792] attempt: attempt_1463520593104_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:53,349 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B3AD3CC7DA4947F4AEE5F954412D8792
2016-05-17 21:29:53,349 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792]
2016-05-17 21:29:53,349 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:53,349 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:TASK_STARTED]: vertexName=B3AD3CC7DA4947F4AEE5F954412D8792, taskId=task_1463520593104_0001_1_01_000000, scheduledTime=1463520593349, launchTime=1463520593349
2016-05-17 21:29:53,349 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520593104_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:53,349 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792]
2016-05-17 21:29:53,350 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8089A5942E8341AE9AB81BEFC77D4422/D44F4D90A5FB4CC99C237E1C8DBC0C58] every/(1/1) ...emblyplatform/ALL_ARGS_-1, VertexName: B3AD3CC7DA4947F4AEE5F954412D8792, VertexParallelism: 1, TaskAttemptID:attempt_1463520593104_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B1666ED4517A412780E5DAC1FCB6D838, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B4BEBB14E0FD42E389F60AA86019AA0A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:53,350 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520593104_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:53,350 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:53,350 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520593104_0001_00_000001, stoppedTime=1463520593350, exitStatus=0
2016-05-17 21:29:53,350 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593104_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:53,350 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520593104_0001_00_000002
2016-05-17 21:29:53,351 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593104_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:53,351 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520593104_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:53,351 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:53,351 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:53,351 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:53,352 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520593104_0001_00_000002
2016-05-17 21:29:53,352 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:53,352 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593104_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:53,354 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593104_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:53,354 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:53,355 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520593104_0001_00_000002, launchTime=1463520593354
2016-05-17 21:29:53,355 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520593104_0001_00_000002
2016-05-17 21:29:53,355 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520593104_0001_1_01_000000_0] to container: [container_1463520593104_0001_00_000002]
2016-05-17 21:29:53,355 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593104_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:53,356 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520593104_0001_1_01_000000_0] started. Is using containerId: [container_1463520593104_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:53,356 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B3AD3CC7DA4947F4AEE5F954412D8792, taskAttemptId=attempt_1463520593104_0001_1_01_000000_0, startTime=1463520593356, containerId=container_1463520593104_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520593104_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:53,356 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:53,356 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520593104_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:53,356 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520593104_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:53,356 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520593104_0001_00_000002 given task: attempt_1463520593104_0001_1_01_000000_0
2016-05-17 21:29:53,356 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520593104_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520593104_0001_1_01_000000_0
2016-05-17 21:29:53 Starting to run new task attempt: attempt_1463520593104_0001_1_01_000000_0
2016-05-17 21:29:53,357 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520593104_0001_1_01_000000_0
2016-05-17 21:29:53,357 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:53,357 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:53,357 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:53,357 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:53,357 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:53,357 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:53,357 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8089A5942E8341AE9AB81BEFC77D4422/D44F4D90A5FB4CC99C237E1C8DBC0C58] every/(1/1) ...emblyplatform/ALL_ARGS_-1, VertexName: B3AD3CC7DA4947F4AEE5F954412D8792, VertexParallelism: 1, TaskAttemptID:attempt_1463520593104_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B1666ED4517A412780E5DAC1FCB6D838, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B4BEBB14E0FD42E389F60AA86019AA0A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:53,358 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:53,358 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520593104_0001_1_01_000000_0
2016-05-17 21:29:53,358 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:53,358 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:53,358 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520593104_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:53,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B1666ED4517A412780E5DAC1FCB6D838, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:53,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B4BEBB14E0FD42E389F60AA86019AA0A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:53,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:53,360 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:53,360 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B4BEBB14E0FD42E389F60AA86019AA0A
2016-05-17 21:29:53,360 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:53,360 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B4BEBB14E0FD42E389F60AA86019AA0A
2016-05-17 21:29:53,359 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520593104_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:53,360 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520593104_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:53,361 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520593104_0001_00_000001, Exiting
2016-05-17 21:29:53,361 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520593104_0001_00_000001
2016-05-17 21:29:53,361 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520593104_0001_00_000001
2016-05-17 21:29:53,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:53,362 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B1666ED4517A412780E5DAC1FCB6D838
2016-05-17 21:29:53,362 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:53,364 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:53,364 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:53,364 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B1666ED4517A412780E5DAC1FCB6D838
2016-05-17 21:29:53,365 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B1666ED4517A412780E5DAC1FCB6D838
2016-05-17 21:29:53,366 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520593104_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:53,366 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520593104_0001_00_000001 completed successfully
2016-05-17 21:29:53,367 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B3AD3CC7DA4947F4AEE5F954412D8792, ordinal: 1
2016-05-17 21:29:53,367 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B3AD3CC7DA4947F4AEE5F954412D8792, mem on start (mb), free: 178, total: 366, max: 672
2016-05-17 21:29:53,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:53,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:53,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:53,367 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593104_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:53,371 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:53,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B4BEBB14E0FD42E389F60AA86019AA0A
2016-05-17 21:29:53,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:53,371 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:53,371 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:53,371 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:53,371 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:53,372 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:53,372 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:53,372 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B4BEBB14E0FD42E389F60AA86019AA0A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:53,372 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B1666ED4517A412780E5DAC1FCB6D838, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:53,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:53,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B1666ED4517A412780E5DAC1FCB6D838 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:53,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:53,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:53,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B1666ED4517A412780E5DAC1FCB6D838
2016-05-17 21:29:53,373 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:53,373 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:53,373 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B1666ED4517A412780E5DAC1FCB6D838 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:53,373 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:53,374 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:53,374 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:53,374 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B1666ED4517A412780E5DAC1FCB6D838
2016-05-17 21:29:53,375 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:53,375 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520593104_0001_1_01_000000_0
2016-05-17 21:29:53,376 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520593104_0001_1_00_000000_0_10321, runDuration: 46788, ]
2016-05-17 21:29:53,377 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520593104_0001_1_00_000000_0_10321] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520593377, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:53,378 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B1666ED4517A412780E5DAC1FCB6D838
2016-05-17 21:29:53,378 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:53,378 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:53,379 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: FE864B2C80CB4907962F27D305625047
2016-05-17 21:29:53,379 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_-1"], id: B4BEBB14E0FD42E389F60AA86019AA0A
2016-05-17 21:29:53,379 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_-1"]
2016-05-17 21:29:53,379 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] FE864B2C80CB4907962F27D305625047, for 1 inputs
2016-05-17 21:29:53,379 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [B1666ED4517A412780E5DAC1FCB6D838] #54 in 2ms
2016-05-17 21:29:53,380 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B1666ED4517A412780E5DAC1FCB6D838] #54, status:false, isInterrupted:false
2016-05-17 21:29:53,380 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:53,381 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:53,381 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:53,381 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:53,382 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:53,382 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:53,382 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B1666ED4517A412780E5DAC1FCB6D838
2016-05-17 21:29:53,382 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:53,382 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B3AD3CC7DA4947F4AEE5F954412D8792, all 1 inputs ready in: 00:00:00.003
2016-05-17 21:29:53,383 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:53,385 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@27f254c6; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@169ba7a7
2016-05-17 21:29:53,385 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:53,389 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:53,392 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:53,392 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:53,393 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520593104_0001_1_01_000000_0
2016-05-17 21:29:53,393 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520593104_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:53,393 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520593104_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:53,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520593382, Process_Duration=11, Process_End_Time=1463520593393, Read_Duration=0, Tuples_Read=1]]
2016-05-17 21:29:53,397 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B3AD3CC7DA4947F4AEE5F954412D8792, mem on close (mb), free: 174, total: 366, max: 672
2016-05-17 21:29:53,398 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:53,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=B3AD3CC7DA4947F4AEE5F954412D8792, index=1
2016-05-17 21:29:53,398 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B3AD3CC7DA4947F4AEE5F954412D8792, taskAttemptId=attempt_1463520593104_0001_1_01_000000_0, startTime=1463520593356, finishTime=1463520593397, timeTaken=41, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520593382, Process_Duration=11, Process_End_Time=1463520593393, Read_Duration=0, Tuples_Read=1
2016-05-17 21:29:53,399 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520593104_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:53,399 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520593104_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:53,399 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:TASK_FINISHED]: vertexName=B3AD3CC7DA4947F4AEE5F954412D8792, taskId=task_1463520593104_0001_1_01_000000, startTime=1463520593356, finishTime=1463520593399, timeTaken=43, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:53,399 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520593104_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:53,399 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:53,399 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:53,399 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792] : 1
2016-05-17 21:29:53,400 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792] because task failed: task_1463520593104_0001_1_01_000000
2016-05-17 21:29:53,400 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:53,400 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:53,400 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792]
2016-05-17 21:29:53,400 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=B4BEBB14E0FD42E389F60AA86019AA0A, vertexId=vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792]
2016-05-17 21:29:53,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:VERTEX_FINISHED]: vertexName=B3AD3CC7DA4947F4AEE5F954412D8792, vertexId=vertex_1463520593104_0001_1_01, initRequestedTime=1463520593263, initedTime=1463520593270, startRequestedTime=1463520593275, startedTime=1463520593275, finishTime=1463520593401, timeTaken=126, status=FAILED, diagnostics=Task failed, taskId=task_1463520593104_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520593356, firstTasksToStart=[ task_1463520593104_0001_1_01_000000 ], lastTaskFinishTime=1463520593397, lastTasksToFinish=[ task_1463520593104_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:53,401 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:53,402 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520593104_0001_00_000002, stoppedTime=1463520593402, exitStatus=0
2016-05-17 21:29:53,402 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593104_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:53,403 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520593104_0001_1_01_000000_0 failed: true container: container_1463520593104_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:53,403 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:53,404 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:53,404 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:53,404 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:53,404 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520593104_0001_1
2016-05-17 21:29:53,404 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520593104_0001_1_00 [B1666ED4517A412780E5DAC1FCB6D838]
2016-05-17 21:29:53,404 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: B4BEBB14E0FD42E389F60AA86019AA0A for vertex: vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792]
2016-05-17 21:29:53,405 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520593104_0001_1, queueSize=0
2016-05-17 21:29:53,405 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:53,406 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593104_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520593104_0001_1, startTime=1463520593261, finishTime=1463520593404, timeTaken=143, status=FAILED, diagnostics=Vertex failed, vertexName=B3AD3CC7DA4947F4AEE5F954412D8792, vertexId=vertex_1463520593104_0001_1_01, diagnostics=[Task failed, taskId=task_1463520593104_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520593104_0001_1_01 [B3AD3CC7DA4947F4AEE5F954412D8792] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520593336, Process_Duration=2, Process_End_Time=1463520593338, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:53,408 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520593104_0001_1 finished with state: FAILED
2016-05-17 21:29:53,408 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520593104_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:53,409 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520593104_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:53 Completed Dag: dag_1463520593104_0001_1
2016-05-17 21:29:53,410 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520593104_0001_1, dagState=FAILED
2016-05-17 21:29:53,410 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520593104_0001_1_post
2016-05-17 21:29:53,412 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:53,412 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:53,412 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593104_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:53,412 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B1666ED4517A412780E5DAC1FCB6D838
2016-05-17 21:29:53,417 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:53,418 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:53,419 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:53,419 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:53,419 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:53,419 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:53,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=B3AD3CC7DA4947F4AEE5F954412D8792, sourceVertex=B1666ED4517A412780E5DAC1FCB6D838
2016-05-17 21:29:53,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=B3AD3CC7DA4947F4AEE5F954412D8792, sourceVertex=B4BEBB14E0FD42E389F60AA86019AA0A
2016-05-17 21:29:53,418 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...emblyplatform/ALL_ARGS_-1, applicationId=application_1463520593104_0001
2016-05-17 21:29:53,421 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520593104_0001_00_000002
2016-05-17 21:29:53,422 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520593104_0001_00_000002
2016-05-17 21:29:53,422 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520593104_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:53,422 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520593104_0001_00_000002 completed but with errors
2016-05-17 21:29:53,423 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:53,423 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:53,423 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:53,423 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:53,423 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:53,423 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520593104_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:53,423 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593104_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:53,423 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520593104_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:53,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:53,424 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:53,424 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:53,425 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:53,425 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:53,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:53,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:53,427 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:53,429 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8089A5942E8341AE9AB81BEFC77D4422/D44F4D90A5FB4CC99C237E1C8DBC0C58/.tez/application_1463520593104_0001
2016-05-17 21:29:53,429 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:53,430 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...emblyplatform/ALL_ARGS_-1
2016-05-17 21:29:53,430 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:53,430 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_-1/_temporary
2016-05-17 21:29:53,431 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.343
2016-05-17 21:29:53,431 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_-1
2016-05-17 21:29:53,453 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:53,454 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:53,486 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 21:29:53,487 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 21:29:53,488 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:53,488 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:53,488 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_'field'
2016-05-17 21:29:53,507 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:53,507 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:53,534 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 21:29:53,539 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-05-17 21:29:53,539 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:53,539 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:53,540 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:53,552 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:53,556 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_RESULTS
2016-05-17 21:29:53,604 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:53,604 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:53,604 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_RESULTS"]
2016-05-17 21:29:53,604 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:53,604 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:53,604 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:53,605 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...platform/ALL_ARGS_RESULTS
2016-05-17 21:29:53,605 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:53,621 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:53,627 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:53,628 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:53,628 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:53,636 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:53,636 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:53,638 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8C46573C12934988B31408FBD0365B5E/D429CD7CC41A40D9B1D3A913A4E29AFF/.tez/application_1463520593627_0001 doesn't exist and is created
2016-05-17 21:29:53,655 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:53,655 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:53,656 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8C46573C12934988B31408FBD0365B5E/D429CD7CC41A40D9B1D3A913A4E29AFF/.tez/application_1463520593627_0001
2016-05-17 21:29:53,658 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520593627_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:53,667 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:53,668 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:53,669 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:53,669 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:53,669 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8C46573C12934988B31408FBD0365B5E/D429CD7CC41A40D9B1D3A913A4E29AFF/.tez/application_1463520593627_0001/localmode-log-dir
2016-05-17 21:29:53,669 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8C46573C12934988B31408FBD0365B5E/D429CD7CC41A40D9B1D3A913A4E29AFF/.tez/application_1463520593627_0001/localmode-log-dir/history.txt.appattempt_1463520593627_0001_000000, maxErrors=10
2016-05-17 21:29:53,670 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:53,670 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520593627_0001_000000, appSubmitTime=1463520593657, launchTime=1463520593657
2016-05-17 21:29:53,675 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:53,675 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:53,676 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:53,676 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:53,677 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:41954
2016-05-17 21:29:53,678 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:53,680 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520593627_0001_000000, startTime=1463520593680
2016-05-17 21:29:53,680 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:53,680 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:53,755 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:53,756 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:53,756 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_RESULTS, applicationId=application_1463520593627_0001, dagName=[8C46573C12934988B31408FBD0365B5E/D429CD7CC41A40D9B1D3A913A4E29AFF] every/(1/1) ...platform/ALL_ARGS_RESULTS
2016-05-17 21:29:53,769 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8C46573C12934988B31408FBD0365B5E/D429CD7CC41A40D9B1D3A913A4E29AFF] every/(1/1) ...platform/ALL_ARGS_RESULTS
2016-05-17 21:29:53,770 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520593627_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8C46573C12934988B31408FBD0365B5E/D429CD7CC41A40D9B1D3A913A4E29AFF/.tez/application_1463520593627_0001/localmode-log-dir/dag_1463520593627_0001_1.dot
2016-05-17 21:29:53,770 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8C46573C12934988B31408FBD0365B5E/D429CD7CC41A40D9B1D3A913A4E29AFF/.tez/application_1463520593627_0001/localmode-log-dir/dag_1463520593627_0001_1-tez-dag.pb.txt
2016-05-17 21:29:53,775 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520593627_0001_1
2016-05-17 21:29:53,775 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:53,776 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8C46573C12934988B31408FBD0365B5E/D429CD7CC41A40D9B1D3A913A4E29AFF] every/(1/1) ...platform/ALL_ARGS_RESULTS
2016-05-17 21:29:53 Running Dag: dag_1463520593627_0001_1
2016-05-17 21:29:53,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:53,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520593627_0001_1, submitTime=1463520593769
2016-05-17 21:29:53,787 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C4163FE751B54F5BADFDB8C8702F31CB
2016-05-17 21:29:53,788 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:53,788 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:53,788 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520593627_0001_1, initTime=1463520593787
2016-05-17 21:29:53,789 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520593627_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:53,789 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:DAG_STARTED]: dagID=dag_1463520593627_0001_1, startTime=1463520593789
2016-05-17 21:29:53,789 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:53,789 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520593627_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:53,789 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: ADA2306B53E143E2A949366B222702C1 : {599D780FD39B44A8BD6AC1F99D14DE19={InputName=599D780FD39B44A8BD6AC1F99D14DE19}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:53,789 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 599D780FD39B44A8BD6AC1F99D14DE19, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:53,790 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1]
2016-05-17 21:29:53,789 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_RESULTS, applicationId=application_1463520593627_0001, dagName=[8C46573C12934988B31408FBD0365B5E/D429CD7CC41A40D9B1D3A913A4E29AFF] every/(1/1) ...platform/ALL_ARGS_RESULTS
2016-05-17 21:29:53,790 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1]
2016-05-17 21:29:53,790 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1]
2016-05-17 21:29:53,790 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1]. Starting root input initializers: 1
2016-05-17 21:29:53,790 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:53,790 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:53,790 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB]
2016-05-17 21:29:53,793 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:53,793 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB]
2016-05-17 21:29:53,793 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB]
2016-05-17 21:29:53,794 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB]
2016-05-17 21:29:53,794 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F11F7D0CBFA64FADB0231B661006B6AD, vertexId=vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:53,794 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F11F7D0CBFA64FADB0231B661006B6AD, vertexId=vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB]
2016-05-17 21:29:53,794 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 599D780FD39B44A8BD6AC1F99D14DE19 on vertex vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1]
2016-05-17 21:29:53,795 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:53,796 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F11F7D0CBFA64FADB0231B661006B6AD, vertexName=C4163FE751B54F5BADFDB8C8702F31CB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:53,796 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F11F7D0CBFA64FADB0231B661006B6AD, vertexId=vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB]
2016-05-17 21:29:53,796 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 599D780FD39B44A8BD6AC1F99D14DE19 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:53,796 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:53,797 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:53,797 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:53,797 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C4163FE751B54F5BADFDB8C8702F31CB, vertexId=vertex_1463520593627_0001_1_01, initRequestedTime=1463520593790, initedTime=1463520593797, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:53,797 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:53,798 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:53,798 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 599D780FD39B44A8BD6AC1F99D14DE19 on vertex vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1]
2016-05-17 21:29:53,799 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:53,799 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {599D780FD39B44A8BD6AC1F99D14DE19=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:53,799 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1] parallelism set to 1
2016-05-17 21:29:53,799 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:53,800 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520593627_0001_1_00, eventCount=1
2016-05-17 21:29:53,800 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ADA2306B53E143E2A949366B222702C1, vertexId=vertex_1463520593627_0001_1_00, initRequestedTime=1463520593789, initedTime=1463520593800, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:53,800 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1] with distanceFromRoot: 0
2016-05-17 21:29:53,800 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:53,800 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:53,801 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in ADA2306B53E143E2A949366B222702C1
2016-05-17 21:29:53,801 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1]
2016-05-17 21:29:53,801 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520593627_0001_1_00, startRequestedTime=1463520593801, startedTime=1463520593801
2016-05-17 21:29:53,801 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:53,801 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:53,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:53,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:TASK_STARTED]: vertexName=ADA2306B53E143E2A949366B222702C1, taskId=task_1463520593627_0001_1_00_000000, scheduledTime=1463520593802, launchTime=1463520593802
2016-05-17 21:29:53,802 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520593627_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:53,802 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520593627_0001_1_00 for vertex: vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB] numStartedSources: 1 numSources: 1
2016-05-17 21:29:53,802 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB] with distanceFromRoot: 1
2016-05-17 21:29:53,803 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1]
2016-05-17 21:29:53,803 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: ADA2306B53E143E2A949366B222702C1 in vertex: C4163FE751B54F5BADFDB8C8702F31CB
2016-05-17 21:29:53,803 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C4163FE751B54F5BADFDB8C8702F31CB with 1 source tasks and 1 pending tasks
2016-05-17 21:29:53,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520593627_0001_1_01, startRequestedTime=1463520593803, startedTime=1463520593803
2016-05-17 21:29:53,803 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:53,804 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:53,804 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8C46573C12934988B31408FBD0365B5E/D429CD7CC41A40D9B1D3A913A4E29AFF] every/(1/1) ...platform/ALL_ARGS_RESULTS, VertexName: ADA2306B53E143E2A949366B222702C1, VertexParallelism: 1, TaskAttemptID:attempt_1463520593627_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=599D780FD39B44A8BD6AC1F99D14DE19, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C4163FE751B54F5BADFDB8C8702F31CB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:53,804 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520593627_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:53,804 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:53,805 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:53,805 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520593627_0001, with dag id: dag_1463520593627_0001_1
2016-05-17 21:29:53,805 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:53,805 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:53,806 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520593627_0001_00_000001
2016-05-17 21:29:53,806 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593627_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:53,807 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520593627_0001_00_000001
2016-05-17 21:29:53,807 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:53,808 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593627_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:53,809 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:53,809 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520593627_0001_00_000001, launchTime=1463520593808
2016-05-17 21:29:53,810 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520593627_0001_00_000001
2016-05-17 21:29:53,810 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520593627_0001_1_00_000000_0] to container: [container_1463520593627_0001_00_000001]
2016-05-17 21:29:53,810 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593627_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:53,811 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520593627_0001_00_000001 given task: attempt_1463520593627_0001_1_00_000000_0
2016-05-17 21:29:53,811 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520593627_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520593627_0001_1_00_000000_0
2016-05-17 21:29:53,811 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520593627_0001_1_00_000000_0] started. Is using containerId: [container_1463520593627_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:53,811 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ADA2306B53E143E2A949366B222702C1, taskAttemptId=attempt_1463520593627_0001_1_00_000000_0, startTime=1463520593811, containerId=container_1463520593627_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520593627_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:53,811 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520593627_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:53,811 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:53,811 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520593627_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:53 Starting to run new task attempt: attempt_1463520593627_0001_1_00_000000_0
2016-05-17 21:29:53,812 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520593627_0001_1_00_000000_0
2016-05-17 21:29:53,812 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:53,812 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:53,812 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:53,812 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:53,813 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:53,813 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:53,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8C46573C12934988B31408FBD0365B5E/D429CD7CC41A40D9B1D3A913A4E29AFF] every/(1/1) ...platform/ALL_ARGS_RESULTS, VertexName: ADA2306B53E143E2A949366B222702C1, VertexParallelism: 1, TaskAttemptID:attempt_1463520593627_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=599D780FD39B44A8BD6AC1F99D14DE19, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C4163FE751B54F5BADFDB8C8702F31CB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:53,813 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:53,814 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520593627_0001_1_00_000000_0
2016-05-17 21:29:53,814 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520593627_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:53,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:53,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:53,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:53,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=599D780FD39B44A8BD6AC1F99D14DE19, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:53,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:53,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 599D780FD39B44A8BD6AC1F99D14DE19
2016-05-17 21:29:53,815 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 599D780FD39B44A8BD6AC1F99D14DE19
2016-05-17 21:29:53,817 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:53,817 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 599D780FD39B44A8BD6AC1F99D14DE19
2016-05-17 21:29:53,817 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:53,817 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 599D780FD39B44A8BD6AC1F99D14DE19
2016-05-17 21:29:53,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C4163FE751B54F5BADFDB8C8702F31CB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:53,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:53,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C4163FE751B54F5BADFDB8C8702F31CB
2016-05-17 21:29:53,819 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:53,819 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C4163FE751B54F5BADFDB8C8702F31CB
2016-05-17 21:29:53,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C4163FE751B54F5BADFDB8C8702F31CB
2016-05-17 21:29:53,821 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ADA2306B53E143E2A949366B222702C1, ordinal: 0
2016-05-17 21:29:53,821 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADA2306B53E143E2A949366B222702C1, mem on start (mb), free: 85, total: 366, max: 672
2016-05-17 21:29:53,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:53,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:53,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:53,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:53,822 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:53,822 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:53,822 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:53,822 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:53,823 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:53,823 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:53,823 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 599D780FD39B44A8BD6AC1F99D14DE19, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:53,823 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C4163FE751B54F5BADFDB8C8702F31CB, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:53,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:53,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 599D780FD39B44A8BD6AC1F99D14DE19 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:53,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 599D780FD39B44A8BD6AC1F99D14DE19
2016-05-17 21:29:53,824 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:53,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 599D780FD39B44A8BD6AC1F99D14DE19
2016-05-17 21:29:53,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:53,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:53,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:53,825 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520593627_0001_1_00_000000_0
2016-05-17 21:29:53,825 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:53,825 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@37024878
2016-05-17 21:29:53,826 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@37024878
2016-05-17 21:29:53,826 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:53,826 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:53,827 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:53,827 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 599D780FD39B44A8BD6AC1F99D14DE19
2016-05-17 21:29:53,827 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C2FA49179B31431C81430B9999B82586
2016-05-17 21:29:53,827 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C2FA49179B31431C81430B9999B82586
2016-05-17 21:29:53,828 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:53,830 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:53,830 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@71801621; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@102d0abe; comparator=cascading.tuple.hadoop.util.TupleComparator@7d1c483d; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:53,830 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:53,830 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:53,831 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:53,857 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:53,857 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:53,857 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:53,857 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:53,857 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:53,858 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:53,858 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:53,858 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ADA2306B53E143E2A949366B222702C1, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:53,860 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520593627_0001_1_00_000000_0
2016-05-17 21:29:53,860 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADA2306B53E143E2A949366B222702C1, mem on close (mb), free: 182, total: 366, max: 672
2016-05-17 21:29:53,860 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:53,860 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:53,860 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:53,860 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:53,863 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:53 Completed running task attempt: attempt_1463520593627_0001_1_00_000000_0
2016-05-17 21:29:53,866 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520593627_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:53,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520593858, Process_Duration=2, Process_End_Time=1463520593860, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:53,866 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520593627_0001_1_00, eventCount=1
2016-05-17 21:29:53,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:53,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ADA2306B53E143E2A949366B222702C1, taskAttemptId=attempt_1463520593627_0001_1_00_000000_0, startTime=1463520593811, finishTime=1463520593866, timeTaken=55, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520593858, Process_Duration=2, Process_End_Time=1463520593860, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:53,867 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520593627_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:53,867 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520593627_0001_1_00_000000_0
2016-05-17 21:29:53,867 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:53,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:TASK_FINISHED]: vertexName=ADA2306B53E143E2A949366B222702C1, taskId=task_1463520593627_0001_1_00_000000, startTime=1463520593811, finishTime=1463520593867, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1463520593627_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520593858, Process_Duration=2, Process_End_Time=1463520593860, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:53,867 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520593627_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:53,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:53,868 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:53,868 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:53,868 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520593627_0001_00_000001
2016-05-17 21:29:53,867 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1] : 1
2016-05-17 21:29:53,868 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1]
2016-05-17 21:29:53,869 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:53,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:VERTEX_FINISHED]: vertexName=ADA2306B53E143E2A949366B222702C1, vertexId=vertex_1463520593627_0001_1_00, initRequestedTime=1463520593789, initedTime=1463520593800, startRequestedTime=1463520593801, startedTime=1463520593801, finishTime=1463520593868, timeTaken=67, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520593858, Process_Duration=2, Process_End_Time=1463520593860, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520593811, firstTasksToStart=[ task_1463520593627_0001_1_00_000000 ], lastTaskFinishTime=1463520593866, lastTasksToFinish=[ task_1463520593627_0001_1_00_000000 ], minTaskDuration=55, maxTaskDuration=55, avgTaskDuration=55.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520593627_0001_1_00_000000 ], longestDurationTasks=[ task_1463520593627_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:53,869 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:53,869 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593627_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:53,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520593627_0001_00_000001, stoppedTime=1463520593869, exitStatus=0
2016-05-17 21:29:53,869 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:53,869 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593627_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:53,870 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB] attempt: attempt_1463520593627_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:53,870 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C4163FE751B54F5BADFDB8C8702F31CB
2016-05-17 21:29:53,870 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB]
2016-05-17 21:29:53,870 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:53,870 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520593627_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:53,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:TASK_STARTED]: vertexName=C4163FE751B54F5BADFDB8C8702F31CB, taskId=task_1463520593627_0001_1_01_000000, scheduledTime=1463520593870, launchTime=1463520593870
2016-05-17 21:29:53,870 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:53,870 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520593627_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:53,871 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593627_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:53,871 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB]
2016-05-17 21:29:53,871 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8C46573C12934988B31408FBD0365B5E/D429CD7CC41A40D9B1D3A913A4E29AFF] every/(1/1) ...platform/ALL_ARGS_RESULTS, VertexName: C4163FE751B54F5BADFDB8C8702F31CB, VertexParallelism: 1, TaskAttemptID:attempt_1463520593627_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ADA2306B53E143E2A949366B222702C1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F11F7D0CBFA64FADB0231B661006B6AD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:53,871 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520593627_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:53,871 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:53,872 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520593627_0001_00_000002
2016-05-17 21:29:53,872 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593627_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:53,873 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520593627_0001_00_000002
2016-05-17 21:29:53,873 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:53,874 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593627_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:53,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520593627_0001_00_000002, launchTime=1463520593874
2016-05-17 21:29:53,874 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:53,875 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520593627_0001_00_000002
2016-05-17 21:29:53,875 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520593627_0001_1_01_000000_0] to container: [container_1463520593627_0001_00_000002]
2016-05-17 21:29:53,875 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593627_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:53,875 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520593627_0001_00_000002 given task: attempt_1463520593627_0001_1_01_000000_0
2016-05-17 21:29:53,875 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520593627_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520593627_0001_1_01_000000_0
2016-05-17 21:29:53,876 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520593627_0001_1_01_000000_0] started. Is using containerId: [container_1463520593627_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:53,876 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:53,876 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C4163FE751B54F5BADFDB8C8702F31CB, taskAttemptId=attempt_1463520593627_0001_1_01_000000_0, startTime=1463520593876, containerId=container_1463520593627_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520593627_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:53,876 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520593627_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:53,877 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520593627_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:53 Starting to run new task attempt: attempt_1463520593627_0001_1_01_000000_0
2016-05-17 21:29:53,877 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520593627_0001_1_01_000000_0
2016-05-17 21:29:53,877 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:53,877 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:53,877 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:53,878 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:53,878 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:53,878 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:53,878 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8C46573C12934988B31408FBD0365B5E/D429CD7CC41A40D9B1D3A913A4E29AFF] every/(1/1) ...platform/ALL_ARGS_RESULTS, VertexName: C4163FE751B54F5BADFDB8C8702F31CB, VertexParallelism: 1, TaskAttemptID:attempt_1463520593627_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ADA2306B53E143E2A949366B222702C1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F11F7D0CBFA64FADB0231B661006B6AD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:53,878 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520593627_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:53,878 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520593627_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:53,878 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:53,879 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520593627_0001_00_000001, Exiting
2016-05-17 21:29:53,879 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520593627_0001_1_01_000000_0
2016-05-17 21:29:53,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:53,879 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520593627_0001_00_000001
2016-05-17 21:29:53,879 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520593627_0001_00_000001
2016-05-17 21:29:53,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:53,880 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520593627_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:53,880 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520593627_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:53,881 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520593627_0001_00_000001 completed successfully
2016-05-17 21:29:53,881 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593627_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:53,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=ADA2306B53E143E2A949366B222702C1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:53,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:53,882 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: ADA2306B53E143E2A949366B222702C1
2016-05-17 21:29:53,882 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:53,882 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F11F7D0CBFA64FADB0231B661006B6AD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:53,883 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:53,883 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F11F7D0CBFA64FADB0231B661006B6AD
2016-05-17 21:29:53,883 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:53,883 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F11F7D0CBFA64FADB0231B661006B6AD
2016-05-17 21:29:53,884 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:53,884 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:53,885 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:53,885 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: ADA2306B53E143E2A949366B222702C1
2016-05-17 21:29:53,885 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: ADA2306B53E143E2A949366B222702C1
2016-05-17 21:29:53,888 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C4163FE751B54F5BADFDB8C8702F31CB, ordinal: 1
2016-05-17 21:29:53,888 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C4163FE751B54F5BADFDB8C8702F31CB, mem on start (mb), free: 176, total: 366, max: 672
2016-05-17 21:29:53,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:53,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:53,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:53,892 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:53,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F11F7D0CBFA64FADB0231B661006B6AD
2016-05-17 21:29:53,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:53,892 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:53,893 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:53,893 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:53,894 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:53,894 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:53,894 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:53,895 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F11F7D0CBFA64FADB0231B661006B6AD, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:53,895 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, ADA2306B53E143E2A949366B222702C1, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:53,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:53,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: ADA2306B53E143E2A949366B222702C1 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:53,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: ADA2306B53E143E2A949366B222702C1
2016-05-17 21:29:53,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:53,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:53,896 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:53,896 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:53,896 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: ADA2306B53E143E2A949366B222702C1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:53,896 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:53,896 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:53,897 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:53,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: ADA2306B53E143E2A949366B222702C1
2016-05-17 21:29:53,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:53,897 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520593627_0001_1_01_000000_0
2016-05-17 21:29:53,899 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520593627_0001_1_00_000000_0_10327, runDuration: 46451, ]
2016-05-17 21:29:53,899 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:53,900 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C2FA49179B31431C81430B9999B82586
2016-05-17 21:29:53,900 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_RESULTS"], id: F11F7D0CBFA64FADB0231B661006B6AD
2016-05-17 21:29:53,900 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_RESULTS"]
2016-05-17 21:29:53,900 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C2FA49179B31431C81430B9999B82586, for 1 inputs
2016-05-17 21:29:53,901 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520593627_0001_1_00_000000_0_10327] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520593901, TimeTaken=1, Rate=0.17 MB/s
2016-05-17 21:29:53,901 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : ADA2306B53E143E2A949366B222702C1
2016-05-17 21:29:53,901 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:53,901 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [ADA2306B53E143E2A949366B222702C1] #55 in 1ms
2016-05-17 21:29:53,902 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [ADA2306B53E143E2A949366B222702C1] #55, status:false, isInterrupted:false
2016-05-17 21:29:53,903 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:53,904 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:53,904 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:53,904 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:53,904 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:53,905 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:53,905 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : ADA2306B53E143E2A949366B222702C1
2016-05-17 21:29:53,905 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C4163FE751B54F5BADFDB8C8702F31CB, all 1 inputs ready in: 00:00:00.004
2016-05-17 21:29:53,906 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:53,907 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:53,908 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@1288f074; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@5c62592d
2016-05-17 21:29:53,908 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:53,910 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:53,910 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:53,910 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:53,910 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520593627_0001_1_01_000000_0
2016-05-17 21:29:53,910 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520593627_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:53,911 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520593627_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:53,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520593906, Process_Duration=5, Process_End_Time=1463520593911, Read_Duration=0, Tuples_Read=1]]
2016-05-17 21:29:53,912 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C4163FE751B54F5BADFDB8C8702F31CB, mem on close (mb), free: 173, total: 366, max: 672
2016-05-17 21:29:53,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=C4163FE751B54F5BADFDB8C8702F31CB, index=1
2016-05-17 21:29:53,912 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: ADA2306B53E143E2A949366B222702C1
2016-05-17 21:29:53,912 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:53,912 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C4163FE751B54F5BADFDB8C8702F31CB, taskAttemptId=attempt_1463520593627_0001_1_01_000000_0, startTime=1463520593876, finishTime=1463520593912, timeTaken=36, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=181, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520593906, Process_Duration=5, Process_End_Time=1463520593911, Read_Duration=0, Tuples_Read=1
2016-05-17 21:29:53,912 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:53,913 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520593627_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:53,913 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520593627_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:53,913 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:TASK_FINISHED]: vertexName=C4163FE751B54F5BADFDB8C8702F31CB, taskId=task_1463520593627_0001_1_01_000000, startTime=1463520593876, finishTime=1463520593913, timeTaken=37, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:53,913 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:53,913 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:53,913 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520593627_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:53,913 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB] : 1
2016-05-17 21:29:53,913 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB] because task failed: task_1463520593627_0001_1_01_000000
2016-05-17 21:29:53,913 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:53,914 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:53,914 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB]
2016-05-17 21:29:53,914 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=F11F7D0CBFA64FADB0231B661006B6AD, vertexId=vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB]
2016-05-17 21:29:53,914 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:53,915 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:VERTEX_FINISHED]: vertexName=C4163FE751B54F5BADFDB8C8702F31CB, vertexId=vertex_1463520593627_0001_1_01, initRequestedTime=1463520593790, initedTime=1463520593797, startRequestedTime=1463520593803, startedTime=1463520593803, finishTime=1463520593915, timeTaken=112, status=FAILED, diagnostics=Task failed, taskId=task_1463520593627_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520593876, firstTasksToStart=[ task_1463520593627_0001_1_01_000000 ], lastTaskFinishTime=1463520593912, lastTasksToFinish=[ task_1463520593627_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:53,915 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:53,915 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:53,915 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520593627_0001_00_000002, stoppedTime=1463520593915, exitStatus=0
2016-05-17 21:29:53,915 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593627_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:53,915 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:53,915 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520593627_0001_1_01_000000_0 failed: true container: container_1463520593627_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:53,916 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:53,916 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:53,916 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520593627_0001_1
2016-05-17 21:29:53,916 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: F11F7D0CBFA64FADB0231B661006B6AD for vertex: vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB]
2016-05-17 21:29:53,916 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:53,916 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:53,916 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520593627_0001_1_00 [ADA2306B53E143E2A949366B222702C1]
2016-05-17 21:29:53,916 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:53,917 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:53,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=C4163FE751B54F5BADFDB8C8702F31CB, sourceVertex=ADA2306B53E143E2A949366B222702C1
2016-05-17 21:29:53,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=C4163FE751B54F5BADFDB8C8702F31CB, sourceVertex=F11F7D0CBFA64FADB0231B661006B6AD
2016-05-17 21:29:53,918 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520593627_0001_1, queueSize=0
2016-05-17 21:29:53,918 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520593627_0001_00_000002
2016-05-17 21:29:53,918 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520593627_0001_00_000002
2016-05-17 21:29:53,918 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:53,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520593627_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520593627_0001_1, startTime=1463520593789, finishTime=1463520593917, timeTaken=128, status=FAILED, diagnostics=Vertex failed, vertexName=C4163FE751B54F5BADFDB8C8702F31CB, vertexId=vertex_1463520593627_0001_1_01, diagnostics=[Task failed, taskId=task_1463520593627_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520593627_0001_1_01 [C4163FE751B54F5BADFDB8C8702F31CB] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520593858, Process_Duration=2, Process_End_Time=1463520593860, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:53,918 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520593627_0001_1 finished with state: FAILED
2016-05-17 21:29:53,918 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520593627_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:53,918 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520593627_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:53,918 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520593627_0001_00_000002 completed but with errors
2016-05-17 21:29:53,919 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520593627_0001_00_000002
2016-05-17 21:29:53 Completed Dag: dag_1463520593627_0001_1
2016-05-17 21:29:53,920 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520593627_0001_1, dagState=FAILED
2016-05-17 21:29:53,920 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520593627_0001_1_post
2016-05-17 21:29:53,920 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:53,920 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:53,922 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520593627_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:53,922 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520593627_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:53,922 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520593627_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:53,922 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_ARGS_RESULTS, applicationId=application_1463520593627_0001
2016-05-17 21:29:53,924 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:53,924 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:53,924 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:53,924 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:53,924 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:53,925 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:53,925 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:53,925 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:53,925 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:53,926 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:53,926 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:53,926 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:53,926 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:53,931 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/8C46573C12934988B31408FBD0365B5E/D429CD7CC41A40D9B1D3A913A4E29AFF/.tez/application_1463520593627_0001
2016-05-17 21:29:53,932 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:53,932 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...platform/ALL_ARGS_RESULTS
2016-05-17 21:29:53,932 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:53,933 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_RESULTS/_temporary
2016-05-17 21:29:53,933 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.328
2016-05-17 21:29:53,933 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_RESULTS
2016-05-17 21:29:53,962 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:53,963 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:53,998 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-05-17 21:29:54,000 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-05-17 21:29:54,000 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:54,000 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-05-17 21:29:54,000 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_ALL
2016-05-17 21:29:54,028 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:54,029 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:54,066 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-05-17 21:29:54,076 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-05-17 21:29:54,077 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:54,077 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:54,077 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:54,095 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:54,101 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_REPLACE
2016-05-17 21:29:54,157 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:54,158 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:54,158 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_REPLACE"]
2016-05-17 21:29:54,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:54,159 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:54,160 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:54,160 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...platform/ALL_ARGS_REPLACE
2016-05-17 21:29:54,161 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:54,170 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:54,174 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:54,174 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:54,175 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:54,175 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:54,175 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:54,185 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1F9C8EF8677D414899F83772A3F35D7C/D75BC761F8E842C6A31CCC2F2927E662/.tez/application_1463520594175_0001 doesn't exist and is created
2016-05-17 21:29:54,199 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:54,199 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:54,199 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1F9C8EF8677D414899F83772A3F35D7C/D75BC761F8E842C6A31CCC2F2927E662/.tez/application_1463520594175_0001
2016-05-17 21:29:54,201 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520594175_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:54,209 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:54,210 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:54,210 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:54,211 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:54,211 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1F9C8EF8677D414899F83772A3F35D7C/D75BC761F8E842C6A31CCC2F2927E662/.tez/application_1463520594175_0001/localmode-log-dir
2016-05-17 21:29:54,211 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1F9C8EF8677D414899F83772A3F35D7C/D75BC761F8E842C6A31CCC2F2927E662/.tez/application_1463520594175_0001/localmode-log-dir/history.txt.appattempt_1463520594175_0001_000000, maxErrors=10
2016-05-17 21:29:54,211 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:54,211 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520594175_0001_000000, appSubmitTime=1463520594200, launchTime=1463520594201
2016-05-17 21:29:54,215 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:54,215 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:54,216 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:56087
2016-05-17 21:29:54,216 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:54,218 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:54,218 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:54,220 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520594175_0001_000000, startTime=1463520594219
2016-05-17 21:29:54,220 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:54,220 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:54,299 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:54,299 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:54,299 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_REPLACE, applicationId=application_1463520594175_0001, dagName=[1F9C8EF8677D414899F83772A3F35D7C/D75BC761F8E842C6A31CCC2F2927E662] every/(1/1) ...platform/ALL_ARGS_REPLACE
2016-05-17 21:29:54,312 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1F9C8EF8677D414899F83772A3F35D7C/D75BC761F8E842C6A31CCC2F2927E662] every/(1/1) ...platform/ALL_ARGS_REPLACE
2016-05-17 21:29:54,313 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520594175_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1F9C8EF8677D414899F83772A3F35D7C/D75BC761F8E842C6A31CCC2F2927E662/.tez/application_1463520594175_0001/localmode-log-dir/dag_1463520594175_0001_1.dot
2016-05-17 21:29:54,314 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1F9C8EF8677D414899F83772A3F35D7C/D75BC761F8E842C6A31CCC2F2927E662/.tez/application_1463520594175_0001/localmode-log-dir/dag_1463520594175_0001_1-tez-dag.pb.txt
2016-05-17 21:29:54,323 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520594175_0001_1
2016-05-17 21:29:54,323 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:54,323 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1F9C8EF8677D414899F83772A3F35D7C/D75BC761F8E842C6A31CCC2F2927E662] every/(1/1) ...platform/ALL_ARGS_REPLACE
2016-05-17 21:29:54 Running Dag: dag_1463520594175_0001_1
2016-05-17 21:29:54,337 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520594175_0001_1, submitTime=1463520594312
2016-05-17 21:29:54,337 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:54,337 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FED41F00C6FF44FAB66EE23668A26363
2016-05-17 21:29:54,337 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:54,338 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520594175_0001_1, initTime=1463520594337
2016-05-17 21:29:54,338 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520594175_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:54,338 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:54,338 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:DAG_STARTED]: dagID=dag_1463520594175_0001_1, startTime=1463520594338
2016-05-17 21:29:54,338 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:54,339 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520594175_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:54,339 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:54,339 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_REPLACE, applicationId=application_1463520594175_0001, dagName=[1F9C8EF8677D414899F83772A3F35D7C/D75BC761F8E842C6A31CCC2F2927E662] every/(1/1) ...platform/ALL_ARGS_REPLACE
2016-05-17 21:29:54,339 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CDC333FE7A39423CAA9642BCF3AD7BD0 : {83C51743206548579F061218591D79F0={InputName=83C51743206548579F061218591D79F0}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:54,340 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 83C51743206548579F061218591D79F0, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:54,340 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0]
2016-05-17 21:29:54,340 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0]
2016-05-17 21:29:54,341 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0]
2016-05-17 21:29:54,341 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0]. Starting root input initializers: 1
2016-05-17 21:29:54,341 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:54,341 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363]
2016-05-17 21:29:54,344 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:54,344 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363]
2016-05-17 21:29:54,344 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363]
2016-05-17 21:29:54,344 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363]
2016-05-17 21:29:54,344 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9EA90018477448B59C1586BB3E764059, vertexId=vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:54,345 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9EA90018477448B59C1586BB3E764059, vertexId=vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363]
2016-05-17 21:29:54,346 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9EA90018477448B59C1586BB3E764059, vertexName=FED41F00C6FF44FAB66EE23668A26363, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:54,346 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9EA90018477448B59C1586BB3E764059, vertexId=vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363]
2016-05-17 21:29:54,347 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:54,347 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FED41F00C6FF44FAB66EE23668A26363, vertexId=vertex_1463520594175_0001_1_01, initRequestedTime=1463520594341, initedTime=1463520594347, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:54,348 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:54,348 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 83C51743206548579F061218591D79F0 on vertex vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0]
2016-05-17 21:29:54,349 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:54,350 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 83C51743206548579F061218591D79F0 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:54,350 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:54,350 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:54,351 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:54,352 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 83C51743206548579F061218591D79F0 on vertex vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0]
2016-05-17 21:29:54,352 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:54,352 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {83C51743206548579F061218591D79F0=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:54,352 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0] parallelism set to 1
2016-05-17 21:29:54,352 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:54,352 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520594175_0001_1_00, eventCount=1
2016-05-17 21:29:54,353 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:54,353 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CDC333FE7A39423CAA9642BCF3AD7BD0, vertexId=vertex_1463520594175_0001_1_00, initRequestedTime=1463520594339, initedTime=1463520594353, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:54,353 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0] with distanceFromRoot: 0
2016-05-17 21:29:54,353 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:54,353 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CDC333FE7A39423CAA9642BCF3AD7BD0
2016-05-17 21:29:54,354 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0]
2016-05-17 21:29:54,354 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:54,354 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520594175_0001_1_00, startRequestedTime=1463520594353, startedTime=1463520594353
2016-05-17 21:29:54,354 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:54,355 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:54,355 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:TASK_STARTED]: vertexName=CDC333FE7A39423CAA9642BCF3AD7BD0, taskId=task_1463520594175_0001_1_00_000000, scheduledTime=1463520594354, launchTime=1463520594354
2016-05-17 21:29:54,355 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520594175_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:54,355 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520594175_0001_1_00 for vertex: vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363] numStartedSources: 1 numSources: 1
2016-05-17 21:29:54,355 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363] with distanceFromRoot: 1
2016-05-17 21:29:54,355 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0]
2016-05-17 21:29:54,356 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CDC333FE7A39423CAA9642BCF3AD7BD0 in vertex: FED41F00C6FF44FAB66EE23668A26363
2016-05-17 21:29:54,356 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520594175_0001, with dag id: dag_1463520594175_0001_1
2016-05-17 21:29:54,356 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FED41F00C6FF44FAB66EE23668A26363 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:54,356 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520594175_0001_1_01, startRequestedTime=1463520594356, startedTime=1463520594356
2016-05-17 21:29:54,356 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:54,356 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:54,357 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1F9C8EF8677D414899F83772A3F35D7C/D75BC761F8E842C6A31CCC2F2927E662] every/(1/1) ...platform/ALL_ARGS_REPLACE, VertexName: CDC333FE7A39423CAA9642BCF3AD7BD0, VertexParallelism: 1, TaskAttemptID:attempt_1463520594175_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=83C51743206548579F061218591D79F0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FED41F00C6FF44FAB66EE23668A26363, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:54,357 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520594175_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:54,357 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:54,358 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:54,358 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:54,358 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:54,358 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520594175_0001_00_000001
2016-05-17 21:29:54,358 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594175_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:54,360 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520594175_0001_00_000001
2016-05-17 21:29:54,360 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:54,361 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594175_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:54,361 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:54,362 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520594175_0001_00_000001, launchTime=1463520594361
2016-05-17 21:29:54,362 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520594175_0001_00_000001
2016-05-17 21:29:54,363 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520594175_0001_1_00_000000_0] to container: [container_1463520594175_0001_00_000001]
2016-05-17 21:29:54,363 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594175_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:54,363 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520594175_0001_1_00_000000_0] started. Is using containerId: [container_1463520594175_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:54,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CDC333FE7A39423CAA9642BCF3AD7BD0, taskAttemptId=attempt_1463520594175_0001_1_00_000000_0, startTime=1463520594363, containerId=container_1463520594175_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520594175_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:54,364 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520594175_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:54,364 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520594175_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:54,364 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:54,364 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520594175_0001_00_000001 given task: attempt_1463520594175_0001_1_00_000000_0
2016-05-17 21:29:54,364 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520594175_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520594175_0001_1_00_000000_0
2016-05-17 21:29:54 Starting to run new task attempt: attempt_1463520594175_0001_1_00_000000_0
2016-05-17 21:29:54,364 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520594175_0001_1_00_000000_0
2016-05-17 21:29:54,365 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:54,365 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:54,365 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:54,365 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:54,365 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:54,365 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:54,365 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1F9C8EF8677D414899F83772A3F35D7C/D75BC761F8E842C6A31CCC2F2927E662] every/(1/1) ...platform/ALL_ARGS_REPLACE, VertexName: CDC333FE7A39423CAA9642BCF3AD7BD0, VertexParallelism: 1, TaskAttemptID:attempt_1463520594175_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=83C51743206548579F061218591D79F0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FED41F00C6FF44FAB66EE23668A26363, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:54,365 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:54,366 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520594175_0001_1_00_000000_0
2016-05-17 21:29:54,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:54,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:54,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=83C51743206548579F061218591D79F0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:54,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:54,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:54,367 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 83C51743206548579F061218591D79F0
2016-05-17 21:29:54,367 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 83C51743206548579F061218591D79F0
2016-05-17 21:29:54,369 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:54,369 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 83C51743206548579F061218591D79F0
2016-05-17 21:29:54,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 83C51743206548579F061218591D79F0
2016-05-17 21:29:54,367 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520594175_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:54,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FED41F00C6FF44FAB66EE23668A26363, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:54,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:54,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FED41F00C6FF44FAB66EE23668A26363
2016-05-17 21:29:54,369 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:54,372 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:54,372 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FED41F00C6FF44FAB66EE23668A26363
2016-05-17 21:29:54,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FED41F00C6FF44FAB66EE23668A26363
2016-05-17 21:29:54,377 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CDC333FE7A39423CAA9642BCF3AD7BD0, ordinal: 0
2016-05-17 21:29:54,378 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CDC333FE7A39423CAA9642BCF3AD7BD0, mem on start (mb), free: 83, total: 366, max: 672
2016-05-17 21:29:54,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:54,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:54,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:54,378 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:54,378 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:54,379 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:54,379 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:54,379 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:54,379 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:54,379 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:54,379 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 83C51743206548579F061218591D79F0, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:54,380 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FED41F00C6FF44FAB66EE23668A26363, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:54,380 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:54,380 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 83C51743206548579F061218591D79F0
2016-05-17 21:29:54,380 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:54,380 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 83C51743206548579F061218591D79F0
2016-05-17 21:29:54,380 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 83C51743206548579F061218591D79F0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:54,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:54,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:54,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:54,381 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520594175_0001_1_00_000000_0
2016-05-17 21:29:54,381 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:54,382 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@54dc3fd3
2016-05-17 21:29:54,382 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@54dc3fd3
2016-05-17 21:29:54,383 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:54,383 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:54,384 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:54,385 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 83C51743206548579F061218591D79F0
2016-05-17 21:29:54,385 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: D5C7DFC899B84700A43B81CD18D46CE0
2016-05-17 21:29:54,385 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] D5C7DFC899B84700A43B81CD18D46CE0
2016-05-17 21:29:54,387 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:54,391 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:54,391 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@285878bc; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@485e9355; comparator=cascading.tuple.hadoop.util.TupleComparator@391ad691; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:54,391 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:54,391 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:54,391 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:54,419 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:54,419 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:54,419 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:54,420 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:54,420 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:54,421 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:54,421 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:54,421 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CDC333FE7A39423CAA9642BCF3AD7BD0, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:54,424 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520594175_0001_1_00_000000_0
2016-05-17 21:29:54,424 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CDC333FE7A39423CAA9642BCF3AD7BD0, mem on close (mb), free: 181, total: 366, max: 672
2016-05-17 21:29:54,424 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:54,425 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:54,425 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:54,425 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:54,429 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:54 Completed running task attempt: attempt_1463520594175_0001_1_00_000000_0
2016-05-17 21:29:54,433 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520594175_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:54,433 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520594421, Process_Duration=3, Process_End_Time=1463520594424, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:54,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520594175_0001_1_00, eventCount=1
2016-05-17 21:29:54,434 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:54,434 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:54,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CDC333FE7A39423CAA9642BCF3AD7BD0, taskAttemptId=attempt_1463520594175_0001_1_00_000000_0, startTime=1463520594363, finishTime=1463520594434, timeTaken=71, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520594421, Process_Duration=3, Process_End_Time=1463520594424, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:54,434 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520594175_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:54,434 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520594175_0001_1_00_000000_0
2016-05-17 21:29:54,434 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520594175_0001_00_000001
2016-05-17 21:29:54,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:TASK_FINISHED]: vertexName=CDC333FE7A39423CAA9642BCF3AD7BD0, taskId=task_1463520594175_0001_1_00_000000, startTime=1463520594363, finishTime=1463520594434, timeTaken=71, status=SUCCEEDED, successfulAttemptID=attempt_1463520594175_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520594421, Process_Duration=3, Process_End_Time=1463520594424, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:54,434 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520594175_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:54,434 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0] : 1
2016-05-17 21:29:54,435 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0]
2016-05-17 21:29:54,435 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:54,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:VERTEX_FINISHED]: vertexName=CDC333FE7A39423CAA9642BCF3AD7BD0, vertexId=vertex_1463520594175_0001_1_00, initRequestedTime=1463520594339, initedTime=1463520594353, startRequestedTime=1463520594353, startedTime=1463520594353, finishTime=1463520594435, timeTaken=82, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520594421, Process_Duration=3, Process_End_Time=1463520594424, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520594363, firstTasksToStart=[ task_1463520594175_0001_1_00_000000 ], lastTaskFinishTime=1463520594434, lastTasksToFinish=[ task_1463520594175_0001_1_00_000000 ], minTaskDuration=71, maxTaskDuration=71, avgTaskDuration=71.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520594175_0001_1_00_000000 ], longestDurationTasks=[ task_1463520594175_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:54,435 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:54,435 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363] attempt: attempt_1463520594175_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:54,435 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FED41F00C6FF44FAB66EE23668A26363
2016-05-17 21:29:54,435 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363]
2016-05-17 21:29:54,435 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594175_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:54,435 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:54,436 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:TASK_STARTED]: vertexName=FED41F00C6FF44FAB66EE23668A26363, taskId=task_1463520594175_0001_1_01_000000, scheduledTime=1463520594436, launchTime=1463520594436
2016-05-17 21:29:54,436 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520594175_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:54,436 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363]
2016-05-17 21:29:54,436 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1F9C8EF8677D414899F83772A3F35D7C/D75BC761F8E842C6A31CCC2F2927E662] every/(1/1) ...platform/ALL_ARGS_REPLACE, VertexName: FED41F00C6FF44FAB66EE23668A26363, VertexParallelism: 1, TaskAttemptID:attempt_1463520594175_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CDC333FE7A39423CAA9642BCF3AD7BD0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9EA90018477448B59C1586BB3E764059, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:54,436 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520594175_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:54,436 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:54,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:54,436 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:54,437 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:54,437 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:54,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520594175_0001_00_000001, stoppedTime=1463520594437, exitStatus=0
2016-05-17 21:29:54,437 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594175_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:54,438 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520594175_0001_00_000002
2016-05-17 21:29:54,438 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594175_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:54,438 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:54,439 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520594175_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:54,440 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520594175_0001_00_000002
2016-05-17 21:29:54,440 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:54,441 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594175_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:54,441 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594175_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:54,441 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:54,441 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520594175_0001_00_000002, launchTime=1463520594441
2016-05-17 21:29:54,442 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520594175_0001_00_000002
2016-05-17 21:29:54,442 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520594175_0001_1_01_000000_0] to container: [container_1463520594175_0001_00_000002]
2016-05-17 21:29:54,442 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594175_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:54,442 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520594175_0001_00_000002 given task: attempt_1463520594175_0001_1_01_000000_0
2016-05-17 21:29:54,443 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520594175_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520594175_0001_1_01_000000_0
2016-05-17 21:29:54,443 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520594175_0001_1_01_000000_0] started. Is using containerId: [container_1463520594175_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:54 Starting to run new task attempt: attempt_1463520594175_0001_1_01_000000_0
2016-05-17 21:29:54,443 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520594175_0001_1_01_000000_0
2016-05-17 21:29:54,443 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:54,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:54,443 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:54,443 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:54,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FED41F00C6FF44FAB66EE23668A26363, taskAttemptId=attempt_1463520594175_0001_1_01_000000_0, startTime=1463520594443, containerId=container_1463520594175_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520594175_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:54,443 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520594175_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:54,443 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:54,444 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:54,444 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:54,444 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1F9C8EF8677D414899F83772A3F35D7C/D75BC761F8E842C6A31CCC2F2927E662] every/(1/1) ...platform/ALL_ARGS_REPLACE, VertexName: FED41F00C6FF44FAB66EE23668A26363, VertexParallelism: 1, TaskAttemptID:attempt_1463520594175_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CDC333FE7A39423CAA9642BCF3AD7BD0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9EA90018477448B59C1586BB3E764059, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:54,444 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:54,445 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520594175_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:54,445 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520594175_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:54,445 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520594175_0001_00_000001, Exiting
2016-05-17 21:29:54,445 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520594175_0001_00_000001
2016-05-17 21:29:54,445 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520594175_0001_00_000001
2016-05-17 21:29:54,444 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520594175_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:54,446 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520594175_0001_1_01_000000_0
2016-05-17 21:29:54,446 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:54,446 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:54,447 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520594175_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:54,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:54,447 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520594175_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:54,448 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520594175_0001_00_000001 completed successfully
2016-05-17 21:29:54,448 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594175_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:54,448 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CDC333FE7A39423CAA9642BCF3AD7BD0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:54,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:54,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CDC333FE7A39423CAA9642BCF3AD7BD0
2016-05-17 21:29:54,450 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:54,450 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:54,451 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:54,451 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CDC333FE7A39423CAA9642BCF3AD7BD0
2016-05-17 21:29:54,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CDC333FE7A39423CAA9642BCF3AD7BD0
2016-05-17 21:29:54,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9EA90018477448B59C1586BB3E764059, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:54,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:54,453 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9EA90018477448B59C1586BB3E764059
2016-05-17 21:29:54,453 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:54,453 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9EA90018477448B59C1586BB3E764059
2016-05-17 21:29:54,456 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FED41F00C6FF44FAB66EE23668A26363, ordinal: 1
2016-05-17 21:29:54,456 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FED41F00C6FF44FAB66EE23668A26363, mem on start (mb), free: 175, total: 366, max: 672
2016-05-17 21:29:54,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:54,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:54,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:54,465 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:54,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9EA90018477448B59C1586BB3E764059
2016-05-17 21:29:54,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:54,465 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:54,465 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:54,466 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:54,466 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:54,466 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:54,466 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:54,466 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CDC333FE7A39423CAA9642BCF3AD7BD0, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:54,466 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9EA90018477448B59C1586BB3E764059, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:54,466 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:54,466 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CDC333FE7A39423CAA9642BCF3AD7BD0 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:54,466 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:54,466 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:54,466 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CDC333FE7A39423CAA9642BCF3AD7BD0
2016-05-17 21:29:54,467 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:54,467 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:54,467 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CDC333FE7A39423CAA9642BCF3AD7BD0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:54,467 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:54,467 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:54,468 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:54,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CDC333FE7A39423CAA9642BCF3AD7BD0
2016-05-17 21:29:54,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:54,469 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520594175_0001_1_01_000000_0
2016-05-17 21:29:54,471 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520594175_0001_1_00_000000_0_10333, runDuration: 60703, ]
2016-05-17 21:29:54,472 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:54,473 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: D5C7DFC899B84700A43B81CD18D46CE0
2016-05-17 21:29:54,473 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_REPLACE"], id: 9EA90018477448B59C1586BB3E764059
2016-05-17 21:29:54,473 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_REPLACE"]
2016-05-17 21:29:54,473 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] D5C7DFC899B84700A43B81CD18D46CE0, for 1 inputs
2016-05-17 21:29:54,474 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520594175_0001_1_00_000000_0_10333] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520594474, TimeTaken=2, Rate=0.09 MB/s
2016-05-17 21:29:54,474 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CDC333FE7A39423CAA9642BCF3AD7BD0
2016-05-17 21:29:54,476 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:54,476 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [CDC333FE7A39423CAA9642BCF3AD7BD0] #56 in 4ms
2016-05-17 21:29:54,477 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CDC333FE7A39423CAA9642BCF3AD7BD0] #56, status:false, isInterrupted:false
2016-05-17 21:29:54,477 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:54,479 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:54,479 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:54,479 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:54,479 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:54,479 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:54,479 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CDC333FE7A39423CAA9642BCF3AD7BD0
2016-05-17 21:29:54,479 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:54,480 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FED41F00C6FF44FAB66EE23668A26363, all 1 inputs ready in: 00:00:00.006
2016-05-17 21:29:54,481 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:54,482 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@91a0067; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@a106edc
2016-05-17 21:29:54,483 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:54,483 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:54,484 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:54,484 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:54,484 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520594175_0001_1_01_000000_0
2016-05-17 21:29:54,484 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520594175_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:54,485 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520594175_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:54,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520594480, Process_Duration=5, Process_End_Time=1463520594485, Read_Duration=0, Tuples_Read=1]]
2016-05-17 21:29:54,496 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FED41F00C6FF44FAB66EE23668A26363, taskAttemptId=attempt_1463520594175_0001_1_01_000000_0, startTime=1463520594443, finishTime=1463520594496, timeTaken=53, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=528, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520594480, Process_Duration=5, Process_End_Time=1463520594485, Read_Duration=0, Tuples_Read=1
2016-05-17 21:29:54,496 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520594175_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:54,497 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520594175_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:54,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:TASK_FINISHED]: vertexName=FED41F00C6FF44FAB66EE23668A26363, taskId=task_1463520594175_0001_1_01_000000, startTime=1463520594443, finishTime=1463520594497, timeTaken=54, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:54,497 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520594175_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:54,497 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:54,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:54,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:54,497 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363] : 1
2016-05-17 21:29:54,498 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363] because task failed: task_1463520594175_0001_1_01_000000
2016-05-17 21:29:54,498 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:54,498 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:54,499 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363]
2016-05-17 21:29:54,499 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=9EA90018477448B59C1586BB3E764059, vertexId=vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363]
2016-05-17 21:29:54,496 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FED41F00C6FF44FAB66EE23668A26363, mem on close (mb), free: 172, total: 366, max: 672
2016-05-17 21:29:54,499 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=FED41F00C6FF44FAB66EE23668A26363, index=1
2016-05-17 21:29:54,499 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CDC333FE7A39423CAA9642BCF3AD7BD0
2016-05-17 21:29:54,500 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:54,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:VERTEX_FINISHED]: vertexName=FED41F00C6FF44FAB66EE23668A26363, vertexId=vertex_1463520594175_0001_1_01, initRequestedTime=1463520594341, initedTime=1463520594347, startRequestedTime=1463520594356, startedTime=1463520594356, finishTime=1463520594500, timeTaken=144, status=FAILED, diagnostics=Task failed, taskId=task_1463520594175_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520594443, firstTasksToStart=[ task_1463520594175_0001_1_01_000000 ], lastTaskFinishTime=1463520594496, lastTasksToFinish=[ task_1463520594175_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:54,500 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:54,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520594175_0001_00_000002, stoppedTime=1463520594500, exitStatus=0
2016-05-17 21:29:54,500 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594175_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:54,500 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520594175_0001_1_01_000000_0 failed: true container: container_1463520594175_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:54,501 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:54,501 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:54,501 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520594175_0001_1
2016-05-17 21:29:54,501 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 9EA90018477448B59C1586BB3E764059 for vertex: vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363]
2016-05-17 21:29:54,501 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520594175_0001_1_00 [CDC333FE7A39423CAA9642BCF3AD7BD0]
2016-05-17 21:29:54,502 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:54,502 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:54,503 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:54,503 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:54,503 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:54,503 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:54,503 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520594175_0001_1, queueSize=0
2016-05-17 21:29:54,503 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:54,504 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=FED41F00C6FF44FAB66EE23668A26363, sourceVertex=CDC333FE7A39423CAA9642BCF3AD7BD0
2016-05-17 21:29:54,504 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=FED41F00C6FF44FAB66EE23668A26363, sourceVertex=9EA90018477448B59C1586BB3E764059
2016-05-17 21:29:54,504 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:54,505 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520594175_0001_00_000002
2016-05-17 21:29:54,505 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520594175_0001_00_000002
2016-05-17 21:29:54,506 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520594175_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:54,506 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520594175_0001_00_000002 completed but with errors
2016-05-17 21:29:54,504 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594175_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520594175_0001_1, startTime=1463520594338, finishTime=1463520594501, timeTaken=163, status=FAILED, diagnostics=Vertex failed, vertexName=FED41F00C6FF44FAB66EE23668A26363, vertexId=vertex_1463520594175_0001_1_01, diagnostics=[Task failed, taskId=task_1463520594175_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520594175_0001_1_01 [FED41F00C6FF44FAB66EE23668A26363] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520594421, Process_Duration=3, Process_End_Time=1463520594424, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:54,506 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520594175_0001_1 finished with state: FAILED
2016-05-17 21:29:54,506 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520594175_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:54,506 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1463520594175_0001_00_000002
2016-05-17 21:29:54,507 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520594175_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:54,507 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594175_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-05-17 21:29:54 Completed Dag: dag_1463520594175_0001_1
2016-05-17 21:29:54,508 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520594175_0001_1, dagState=FAILED
2016-05-17 21:29:54,508 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520594175_0001_1_post
2016-05-17 21:29:54,508 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:54,508 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:54,508 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520594175_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:54,510 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_ARGS_REPLACE, applicationId=application_1463520594175_0001
2016-05-17 21:29:54,512 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:54,512 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:54,512 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:54,512 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:54,512 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:54,516 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:54,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:54,517 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:54,517 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:54,517 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:54,518 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:54,518 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:54,518 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:54,522 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/1F9C8EF8677D414899F83772A3F35D7C/D75BC761F8E842C6A31CCC2F2927E662/.tez/application_1463520594175_0001
2016-05-17 21:29:54,522 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:54,522 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...platform/ALL_ARGS_REPLACE
2016-05-17 21:29:54,523 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:54,523 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_REPLACE/_temporary
2016-05-17 21:29:54,523 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.366
2016-05-17 21:29:54,523 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_REPLACE
2016-05-17 21:29:54,552 INFO util.Util (Util.java:findMainClass(1374)) - using default application jar, may cause class not found exceptions on the cluster
2016-05-17 21:29:54,552 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(151)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.1.0/226af02e24d5195c60e4ebe911d0c7c9780f59f0/cascading-hadoop2-tez-3.1.0.jar
2016-05-17 21:29:54,591 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-05-17 21:29:54,602 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-05-17 21:29:54,603 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:54,603 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-05-17 21:29:54,603 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-05-17 21:29:54,615 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-05-17 21:29:54,618 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_SWAP
2016-05-17 21:29:54,655 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting
2016-05-17 21:29:54,655 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:54,655 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] sink: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_SWAP"]
2016-05-17 21:29:54,657 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] parallel execution of steps is enabled: false
2016-05-17 21:29:54,657 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] executing total steps: 1
2016-05-17 21:29:54,657 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] allocating management threads: 1
2016-05-17 21:29:54,657 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] starting step: (1/1) ...blyplatform/ALL_ARGS_SWAP
2016-05-17 21:29:54,658 WARN flow.Flow (BaseFlow.java:logWarn(1516)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-05-17 21:29:54,667 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] tez session mode enabled: true
2016-05-17 21:29:54,671 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:54,672 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-05-17 21:29:54,672 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-05-17 21:29:54,673 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-05-17 21:29:54,673 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-05-17 21:29:54,681 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E3753DA17CE04D928BBD580BE2C1220C/E3CFB70A618241E1823F8AB2D7C8BE36/.tez/application_1463520594671_0001 doesn't exist and is created
2016-05-17 21:29:54,695 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-05-17 21:29:54,695 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-05-17 21:29:54,696 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E3753DA17CE04D928BBD580BE2C1220C/E3CFB70A618241E1823F8AB2D7C8BE36/.tez/application_1463520594671_0001
2016-05-17 21:29:54,699 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1463520594671_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-05-17 21:29:54,708 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-05-17 21:29:54,709 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-05-17 21:29:54,709 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-05-17 21:29:54,710 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-05-17 21:29:54,710 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E3753DA17CE04D928BBD580BE2C1220C/E3CFB70A618241E1823F8AB2D7C8BE36/.tez/application_1463520594671_0001/localmode-log-dir
2016-05-17 21:29:54,712 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E3753DA17CE04D928BBD580BE2C1220C/E3CFB70A618241E1823F8AB2D7C8BE36/.tez/application_1463520594671_0001/localmode-log-dir/history.txt.appattempt_1463520594671_0001_000000, maxErrors=10
2016-05-17 21:29:54,712 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-05-17 21:29:54,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1463520594671_0001_000000, appSubmitTime=1463520594699, launchTime=1463520594699
2016-05-17 21:29:54,717 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-123-148-208.ec2.internal/10.123.148.208:35521
2016-05-17 21:29:54,720 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-05-17 21:29:54,720 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-05-17 21:29:54,721 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-05-17 21:29:54,721 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-05-17 21:29:54,723 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-05-17 21:29:54,724 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1463520594671_0001_000000, startTime=1463520594724
2016-05-17 21:29:54,724 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-05-17 21:29:54,724 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-05-17 21:29:54,796 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-05-17 21:29:54,796 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-05-17 21:29:54,796 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_ARGS_SWAP, applicationId=application_1463520594671_0001, dagName=[E3753DA17CE04D928BBD580BE2C1220C/E3CFB70A618241E1823F8AB2D7C8BE36] every/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-05-17 21:29:54,809 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [E3753DA17CE04D928BBD580BE2C1220C/E3CFB70A618241E1823F8AB2D7C8BE36] every/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-05-17 21:29:54,810 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1463520594671_0001_1, filePath=/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E3753DA17CE04D928BBD580BE2C1220C/E3CFB70A618241E1823F8AB2D7C8BE36/.tez/application_1463520594671_0001/localmode-log-dir/dag_1463520594671_0001_1.dot
2016-05-17 21:29:54,810 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E3753DA17CE04D928BBD580BE2C1220C/E3CFB70A618241E1823F8AB2D7C8BE36/.tez/application_1463520594671_0001/localmode-log-dir/dag_1463520594671_0001_1-tez-dag.pb.txt
2016-05-17 21:29:54,815 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520594671_0001_1
2016-05-17 21:29:54,815 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:54,816 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [E3753DA17CE04D928BBD580BE2C1220C/E3CFB70A618241E1823F8AB2D7C8BE36] every/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-05-17 21:29:54 Running Dag: dag_1463520594671_0001_1
2016-05-17 21:29:54,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1463520594671_0001_1, submitTime=1463520594809
2016-05-17 21:29:54,825 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-05-17 21:29:54,826 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B73C7AD560684BFDB1B5AE04B2DD7C77
2016-05-17 21:29:54,826 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-05-17 21:29:54,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1463520594671_0001_1, initTime=1463520594825
2016-05-17 21:29:54,826 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-05-17 21:29:54,826 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520594671_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-05-17 21:29:54,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:DAG_STARTED]: dagID=dag_1463520594671_0001_1, startTime=1463520594827
2016-05-17 21:29:54,827 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-05-17 21:29:54,827 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520594671_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-05-17 21:29:54,827 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-05-17 21:29:54,827 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CE5A74EB17C0417CBAAB125A95AF7BF8 : {D4C20E78F51E40EAA0D6BDF7D69923E5={InputName=D4C20E78F51E40EAA0D6BDF7D69923E5}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-05-17 21:29:54,827 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: D4C20E78F51E40EAA0D6BDF7D69923E5, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-05-17 21:29:54,827 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8]
2016-05-17 21:29:54,827 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_ARGS_SWAP, applicationId=application_1463520594671_0001, dagName=[E3753DA17CE04D928BBD580BE2C1220C/E3CFB70A618241E1823F8AB2D7C8BE36] every/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-05-17 21:29:54,827 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8]
2016-05-17 21:29:54,828 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8]
2016-05-17 21:29:54,828 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8]. Starting root input initializers: 1
2016-05-17 21:29:54,828 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8] transitioned from NEW to INITIALIZING due to event V_INIT
2016-05-17 21:29:54,828 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77]
2016-05-17 21:29:54,828 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: D4C20E78F51E40EAA0D6BDF7D69923E5 on vertex vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8]
2016-05-17 21:29:54,830 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-05-17 21:29:54,830 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input D4C20E78F51E40EAA0D6BDF7D69923E5 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-05-17 21:29:54,830 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-05-17 21:29:54,830 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-05-17 21:29:54,831 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-05-17 21:29:54,831 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77]
2016-05-17 21:29:54,832 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77]
2016-05-17 21:29:54,832 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77]
2016-05-17 21:29:54,832 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=F291B28A9E234D1491D4E8F1231CE331, vertexId=vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-05-17 21:29:54,832 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-05-17 21:29:54,832 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=F291B28A9E234D1491D4E8F1231CE331, vertexId=vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77]
2016-05-17 21:29:54,832 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: D4C20E78F51E40EAA0D6BDF7D69923E5 on vertex vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8]
2016-05-17 21:29:54,834 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=F291B28A9E234D1491D4E8F1231CE331, vertexName=B73C7AD560684BFDB1B5AE04B2DD7C77, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-05-17 21:29:54,834 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=F291B28A9E234D1491D4E8F1231CE331, vertexId=vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77]
2016-05-17 21:29:54,835 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:54,835 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B73C7AD560684BFDB1B5AE04B2DD7C77, vertexId=vertex_1463520594671_0001_1_01, initRequestedTime=1463520594828, initedTime=1463520594835, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-05-17 21:29:54,835 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77] transitioned from NEW to INITED due to event V_INIT
2016-05-17 21:29:54,835 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:54,836 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {D4C20E78F51E40EAA0D6BDF7D69923E5=forAllWorkUnits=true, update=[1]}
2016-05-17 21:29:54,836 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8] parallelism set to 1
2016-05-17 21:29:54,836 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-05-17 21:29:54,836 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520594671_0001_1_00, eventCount=1
2016-05-17 21:29:54,837 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-05-17 21:29:54,837 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CE5A74EB17C0417CBAAB125A95AF7BF8, vertexId=vertex_1463520594671_0001_1_00, initRequestedTime=1463520594827, initedTime=1463520594836, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-05-17 21:29:54,837 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8] with distanceFromRoot: 0
2016-05-17 21:29:54,837 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-05-17 21:29:54,837 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CE5A74EB17C0417CBAAB125A95AF7BF8
2016-05-17 21:29:54,837 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8]
2016-05-17 21:29:54,838 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520594671_0001_1_00, startRequestedTime=1463520594837, startedTime=1463520594837
2016-05-17 21:29:54,838 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:54,838 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:54,838 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:54,838 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:TASK_STARTED]: vertexName=CE5A74EB17C0417CBAAB125A95AF7BF8, taskId=task_1463520594671_0001_1_00_000000, scheduledTime=1463520594838, launchTime=1463520594838
2016-05-17 21:29:54,839 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520594671_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:54,839 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1463520594671_0001_1_00 for vertex: vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77] numStartedSources: 1 numSources: 1
2016-05-17 21:29:54,839 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77] with distanceFromRoot: 1
2016-05-17 21:29:54,839 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8]
2016-05-17 21:29:54,839 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CE5A74EB17C0417CBAAB125A95AF7BF8 in vertex: B73C7AD560684BFDB1B5AE04B2DD7C77
2016-05-17 21:29:54,840 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B73C7AD560684BFDB1B5AE04B2DD7C77 with 1 source tasks and 1 pending tasks
2016-05-17 21:29:54,840 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1463520594671_0001_1_01, startRequestedTime=1463520594839, startedTime=1463520594839
2016-05-17 21:29:54,840 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-05-17 21:29:54,840 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77] transitioned from INITED to RUNNING due to event V_START
2016-05-17 21:29:54,840 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E3753DA17CE04D928BBD580BE2C1220C/E3CFB70A618241E1823F8AB2D7C8BE36] every/(1/1) ...blyplatform/ALL_ARGS_SWAP, VertexName: CE5A74EB17C0417CBAAB125A95AF7BF8, VertexParallelism: 1, TaskAttemptID:attempt_1463520594671_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D4C20E78F51E40EAA0D6BDF7D69923E5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B73C7AD560684BFDB1B5AE04B2DD7C77, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:54,841 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520594671_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:54,841 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:54,841 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-05-17 21:29:54,842 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-05-17 21:29:54,842 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-05-17 21:29:54,842 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520594671_0001_00_000001
2016-05-17 21:29:54,842 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594671_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:54,844 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520594671_0001_00_000001
2016-05-17 21:29:54,844 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:54,844 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] submitted tez dag to app master: application_1463520594671_0001, with dag id: dag_1463520594671_0001_1
2016-05-17 21:29:54,844 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594671_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:54,844 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:54,845 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520594671_0001_00_000001, launchTime=1463520594844
2016-05-17 21:29:54,845 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520594671_0001_00_000001
2016-05-17 21:29:54,846 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520594671_0001_1_00_000000_0] to container: [container_1463520594671_0001_00_000001]
2016-05-17 21:29:54,846 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594671_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:54,846 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520594671_0001_1_00_000000_0] started. Is using containerId: [container_1463520594671_0001_00_000001] on NM: [127.0.0.1:0]
2016-05-17 21:29:54,847 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:54,847 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520594671_0001_00_000001 given task: attempt_1463520594671_0001_1_00_000000_0
2016-05-17 21:29:54,847 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CE5A74EB17C0417CBAAB125A95AF7BF8, taskAttemptId=attempt_1463520594671_0001_1_00_000000_0, startTime=1463520594846, containerId=container_1463520594671_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520594671_0001_00_000001/teamcity, completedLogs=
2016-05-17 21:29:54,847 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520594671_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:54,847 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520594671_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:54,847 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520594671_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520594671_0001_1_00_000000_0
2016-05-17 21:29:54 Starting to run new task attempt: attempt_1463520594671_0001_1_00_000000_0
2016-05-17 21:29:54,847 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520594671_0001_1_00_000000_0
2016-05-17 21:29:54,848 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:54,848 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:54,848 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:54,848 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:54,848 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:54,848 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:54,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E3753DA17CE04D928BBD580BE2C1220C/E3CFB70A618241E1823F8AB2D7C8BE36] every/(1/1) ...blyplatform/ALL_ARGS_SWAP, VertexName: CE5A74EB17C0417CBAAB125A95AF7BF8, VertexParallelism: 1, TaskAttemptID:attempt_1463520594671_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D4C20E78F51E40EAA0D6BDF7D69923E5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B73C7AD560684BFDB1B5AE04B2DD7C77, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-05-17 21:29:54,848 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:54,849 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520594671_0001_1_00_000000_0
2016-05-17 21:29:54,849 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:54,849 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:54,849 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520594671_0001_1_00_000000_0 sent events: (0-1)
2016-05-17 21:29:54,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:54,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B73C7AD560684BFDB1B5AE04B2DD7C77, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-05-17 21:29:54,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:54,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B73C7AD560684BFDB1B5AE04B2DD7C77
2016-05-17 21:29:54,851 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:54,852 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-05-17 21:29:54,852 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B73C7AD560684BFDB1B5AE04B2DD7C77
2016-05-17 21:29:54,852 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B73C7AD560684BFDB1B5AE04B2DD7C77
2016-05-17 21:29:54,854 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CE5A74EB17C0417CBAAB125A95AF7BF8, ordinal: 0
2016-05-17 21:29:54,854 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE5A74EB17C0417CBAAB125A95AF7BF8, mem on start (mb), free: 97, total: 366, max: 672
2016-05-17 21:29:54,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:54,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:54,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:54,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D4C20E78F51E40EAA0D6BDF7D69923E5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-05-17 21:29:54,854 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:54,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D4C20E78F51E40EAA0D6BDF7D69923E5
2016-05-17 21:29:54,855 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: D4C20E78F51E40EAA0D6BDF7D69923E5
2016-05-17 21:29:54,856 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-05-17 21:29:54,856 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: D4C20E78F51E40EAA0D6BDF7D69923E5
2016-05-17 21:29:54,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D4C20E78F51E40EAA0D6BDF7D69923E5
2016-05-17 21:29:54,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:54,856 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:54,857 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-05-17 21:29:54,857 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-05-17 21:29:54,857 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:54,857 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:54,857 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-05-17 21:29:54,857 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B73C7AD560684BFDB1B5AE04B2DD7C77, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-05-17 21:29:54,857 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D4C20E78F51E40EAA0D6BDF7D69923E5, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-05-17 21:29:54,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:54,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D4C20E78F51E40EAA0D6BDF7D69923E5 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:54,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:54,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:54,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D4C20E78F51E40EAA0D6BDF7D69923E5
2016-05-17 21:29:54,858 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:54,858 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D4C20E78F51E40EAA0D6BDF7D69923E5
2016-05-17 21:29:54,858 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:54,858 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520594671_0001_1_00_000000_0
2016-05-17 21:29:54,860 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-05-17 21:29:54,860 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:54,860 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@773d34fd
2016-05-17 21:29:54,860 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: D4C20E78F51E40EAA0D6BDF7D69923E5
2016-05-17 21:29:54,861 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 216A24F24B6B47388E792E47FD95BF23
2016-05-17 21:29:54,861 INFO element.TezGroupGate (TezGroupGate.java:prepare(98)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 216A24F24B6B47388E792E47FD95BF23
2016-05-17 21:29:54,861 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@773d34fd
2016-05-17 21:29:54,861 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-05-17 21:29:54,861 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-05-17 21:29:54,862 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:54,864 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:54,864 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.KeyTupleSerializer@65e99801; valueSerializer=cascading.tuple.hadoop.io.ValueTupleSerializer@4afa9dec; comparator=cascading.tuple.hadoop.util.TupleComparator@b961769; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-05-17 21:29:54,864 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-05-17 21:29:54,864 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-05-17 21:29:54,864 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:54,885 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-05-17 21:29:54,885 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-05-17 21:29:54,885 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-05-17 21:29:54,885 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-05-17 21:29:54,885 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-05-17 21:29:54,886 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-05-17 21:29:54,886 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-05-17 21:29:54,886 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CE5A74EB17C0417CBAAB125A95AF7BF8, all 1 inputs ready in: 00:00:00.000
2016-05-17 21:29:54,888 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1463520594671_0001_1_00_000000_0
2016-05-17 21:29:54,889 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE5A74EB17C0417CBAAB125A95AF7BF8, mem on close (mb), free: 180, total: 366, max: 672
2016-05-17 21:29:54,889 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-05-17 21:29:54,889 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-05-17 21:29:54,889 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 131; bufvoid = 104857600
2016-05-17 21:29:54,889 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-05-17 21:29:54,892 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-05-17 21:29:54 Completed running task attempt: attempt_1463520594671_0001_1_00_000000_0
2016-05-17 21:29:54,894 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1463520594671_0001_1_00_000000_0, fatalErrorOccurred=false
2016-05-17 21:29:54,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1463520594886, Process_Duration=2, Process_End_Time=1463520594888, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-05-17 21:29:54,895 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1463520594671_0001_1_00, eventCount=1
2016-05-17 21:29:54,895 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-05-17 21:29:54,895 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:54,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:54,895 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CE5A74EB17C0417CBAAB125A95AF7BF8, taskAttemptId=attempt_1463520594671_0001_1_00_000000_0, startTime=1463520594846, finishTime=1463520594895, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520594886, Process_Duration=2, Process_End_Time=1463520594888, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:54,895 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520594671_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-05-17 21:29:54,895 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:54,895 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1463520594671_0001_1_00_000000_0
2016-05-17 21:29:54,895 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520594671_0001_00_000001
2016-05-17 21:29:54,896 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:54,896 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:TASK_FINISHED]: vertexName=CE5A74EB17C0417CBAAB125A95AF7BF8, taskId=task_1463520594671_0001_1_00_000000, startTime=1463520594846, finishTime=1463520594896, timeTaken=50, status=SUCCEEDED, successfulAttemptID=attempt_1463520594671_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520594886, Process_Duration=2, Process_End_Time=1463520594888, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:54,896 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520594671_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-05-17 21:29:54,896 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8] : 1
2016-05-17 21:29:54,896 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8]
2016-05-17 21:29:54,897 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:54,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:VERTEX_FINISHED]: vertexName=CE5A74EB17C0417CBAAB125A95AF7BF8, vertexId=vertex_1463520594671_0001_1_00, initRequestedTime=1463520594827, initedTime=1463520594836, startRequestedTime=1463520594837, startedTime=1463520594837, finishTime=1463520594896, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520594886, Process_Duration=2, Process_End_Time=1463520594888, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1463520594846, firstTasksToStart=[ task_1463520594671_0001_1_00_000000 ], lastTaskFinishTime=1463520594895, lastTasksToFinish=[ task_1463520594671_0001_1_00_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1463520594671_0001_1_00_000000 ], longestDurationTasks=[ task_1463520594671_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-05-17 21:29:54,897 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-05-17 21:29:54,897 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594671_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-05-17 21:29:54,897 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77] attempt: attempt_1463520594671_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-05-17 21:29:54,897 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B73C7AD560684BFDB1B5AE04B2DD7C77
2016-05-17 21:29:54,897 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77]
2016-05-17 21:29:54,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520594671_0001_00_000001, stoppedTime=1463520594898, exitStatus=0
2016-05-17 21:29:54,898 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594671_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:54,898 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-05-17 21:29:54,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:TASK_STARTED]: vertexName=B73C7AD560684BFDB1B5AE04B2DD7C77, taskId=task_1463520594671_0001_1_01_000000, scheduledTime=1463520594898, launchTime=1463520594898
2016-05-17 21:29:54,898 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520594671_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-05-17 21:29:54,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:54,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-05-17 21:29:54,899 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520594671_0001_00_000001. Relying on regular task shutdown for it to end
2016-05-17 21:29:54,899 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77]
2016-05-17 21:29:54,899 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594671_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:54,899 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E3753DA17CE04D928BBD580BE2C1220C/E3CFB70A618241E1823F8AB2D7C8BE36] every/(1/1) ...blyplatform/ALL_ARGS_SWAP, VertexName: B73C7AD560684BFDB1B5AE04B2DD7C77, VertexParallelism: 1, TaskAttemptID:attempt_1463520594671_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CE5A74EB17C0417CBAAB125A95AF7BF8, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F291B28A9E234D1491D4E8F1231CE331, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:54,899 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520594671_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-05-17 21:29:54,899 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-05-17 21:29:54,900 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1463520594671_0001_00_000002
2016-05-17 21:29:54,900 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594671_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-05-17 21:29:54,901 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1463520594671_0001_00_000002
2016-05-17 21:29:54,901 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(144)) - DAGAppMaster metrics system already initialized!
2016-05-17 21:29:54,902 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594671_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-05-17 21:29:54,902 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1463520594671_0001_00_000002, launchTime=1463520594902
2016-05-17 21:29:54,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-05-17 21:29:54,903 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1463520594671_0001_00_000002
2016-05-17 21:29:54,904 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1463520594671_0001_1_01_000000_0] to container: [container_1463520594671_0001_00_000002]
2016-05-17 21:29:54,904 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594671_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-05-17 21:29:54,904 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1463520594671_0001_00_000002 given task: attempt_1463520594671_0001_1_01_000000_0
2016-05-17 21:29:54,904 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520594671_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1463520594671_0001_1_01_000000_0
2016-05-17 21:29:54,904 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1463520594671_0001_1_01_000000_0] started. Is using containerId: [container_1463520594671_0001_00_000002] on NM: [127.0.0.1:0]
2016-05-17 21:29:54,904 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-05-17 21:29:54 Starting to run new task attempt: attempt_1463520594671_0001_1_01_000000_0
2016-05-17 21:29:54,904 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1463520594671_0001_1_01_000000_0
2016-05-17 21:29:54,905 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:54,905 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-05-17 21:29:54,905 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-05-17 21:29:54,905 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B73C7AD560684BFDB1B5AE04B2DD7C77, taskAttemptId=attempt_1463520594671_0001_1_01_000000_0, startTime=1463520594904, containerId=container_1463520594671_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1463520594671_0001_00_000002/teamcity, completedLogs=
2016-05-17 21:29:54,905 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-05-17 21:29:54,905 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-05-17 21:29:54,905 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520594671_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-05-17 21:29:54,905 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-05-17 21:29:54,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E3753DA17CE04D928BBD580BE2C1220C/E3CFB70A618241E1823F8AB2D7C8BE36] every/(1/1) ...blyplatform/ALL_ARGS_SWAP, VertexName: B73C7AD560684BFDB1B5AE04B2DD7C77, VertexParallelism: 1, TaskAttemptID:attempt_1463520594671_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CE5A74EB17C0417CBAAB125A95AF7BF8, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=F291B28A9E234D1491D4E8F1231CE331, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-05-17 21:29:54,906 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-05-17 21:29:54,905 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520594671_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-05-17 21:29:54,906 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1463520594671_0001_1_01_000000_0
2016-05-17 21:29:54,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-05-17 21:29:54,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:54,907 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1463520594671_0001_1_01_000000_0 sent events: (0-1)
2016-05-17 21:29:54,907 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1463520594671_0001_00_000001 is valid, but no longer registered, and will be killed
2016-05-17 21:29:54,907 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1463520594671_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-05-17 21:29:54,907 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1463520594671_0001_00_000001, Exiting
2016-05-17 21:29:54,907 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520594671_0001_00_000001
2016-05-17 21:29:54,907 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520594671_0001_00_000001
2016-05-17 21:29:54,907 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CE5A74EB17C0417CBAAB125A95AF7BF8, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-05-17 21:29:54,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-05-17 21:29:54,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CE5A74EB17C0417CBAAB125A95AF7BF8
2016-05-17 21:29:54,907 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:54,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F291B28A9E234D1491D4E8F1231CE331, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-05-17 21:29:54,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-05-17 21:29:54,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F291B28A9E234D1491D4E8F1231CE331
2016-05-17 21:29:54,910 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-05-17 21:29:54,910 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: F291B28A9E234D1491D4E8F1231CE331
2016-05-17 21:29:54,911 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-05-17 21:29:54,911 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-05-17 21:29:54,911 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CE5A74EB17C0417CBAAB125A95AF7BF8
2016-05-17 21:29:54,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CE5A74EB17C0417CBAAB125A95AF7BF8
2016-05-17 21:29:54,912 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.1.0
2016-05-17 21:29:54,913 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520594671_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-05-17 21:29:54,913 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1463520594671_0001_00_000001 completed successfully
2016-05-17 21:29:54,913 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594671_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:54,917 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B73C7AD560684BFDB1B5AE04B2DD7C77, ordinal: 1
2016-05-17 21:29:54,917 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B73C7AD560684BFDB1B5AE04B2DD7C77, mem on start (mb), free: 174, total: 366, max: 672
2016-05-17 21:29:54,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-05-17 21:29:54,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-05-17 21:29:54,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-05-17 21:29:54,920 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-05-17 21:29:54,921 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F291B28A9E234D1491D4E8F1231CE331
2016-05-17 21:29:54,921 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-05-17 21:29:54,921 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-05-17 21:29:54,921 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-05-17 21:29:54,921 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-05-17 21:29:54,921 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-05-17 21:29:54,921 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-05-17 21:29:54,921 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-05-17 21:29:54,921 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F291B28A9E234D1491D4E8F1231CE331, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-05-17 21:29:54,922 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CE5A74EB17C0417CBAAB125A95AF7BF8, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-05-17 21:29:54,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-05-17 21:29:54,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CE5A74EB17C0417CBAAB125A95AF7BF8
2016-05-17 21:29:54,922 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-05-17 21:29:54,922 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-05-17 21:29:54,923 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CE5A74EB17C0417CBAAB125A95AF7BF8 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-05-17 21:29:54,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CE5A74EB17C0417CBAAB125A95AF7BF8 being auto started by the framework. Subsequent instances will not be auto-started
2016-05-17 21:29:54,923 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-05-17 21:29:54,923 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-05-17 21:29:54,923 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-05-17 21:29:54,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CE5A74EB17C0417CBAAB125A95AF7BF8
2016-05-17 21:29:54,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-05-17 21:29:54,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-05-17 21:29:54,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-05-17 21:29:54,924 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1463520594671_0001_1_01_000000_0
2016-05-17 21:29:54,925 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-123-148-208, port: 0, pathComponent: attempt_1463520594671_0001_1_00_000000_0_10338, runDuration: 42759, ]
2016-05-17 21:29:54,926 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1463520594671_0001_1_00_000000_0_10338] to DISK_DIRECT, CompressedSize=181, DecompressedSize=177,EndTime=1463520594926, TimeTaken=0, Rate=0.00 MB/s
2016-05-17 21:29:54,926 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CE5A74EB17C0417CBAAB125A95AF7BF8
2016-05-17 21:29:54,926 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-05-17 21:29:54,927 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-123-148-208:0 freed by fetcher [CE5A74EB17C0417CBAAB125A95AF7BF8] #57 in 1ms
2016-05-17 21:29:54,927 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CE5A74EB17C0417CBAAB125A95AF7BF8] #57, status:false, isInterrupted:false
2016-05-17 21:29:54,927 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-05-17 21:29:54,928 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 216A24F24B6B47388E792E47FD95BF23
2016-05-17 21:29:54,928 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_SWAP"], id: F291B28A9E234D1491D4E8F1231CE331
2016-05-17 21:29:54,928 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line' | long, String]->[ALL]]"]["/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_SWAP"]
2016-05-17 21:29:54,927 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-05-17 21:29:54,928 INFO element.TezGroupGate (TezGroupGate.java:prepare(90)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 216A24F24B6B47388E792E47FD95BF23, for 1 inputs
2016-05-17 21:29:54,929 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:54,929 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 181 bytes from disk
2016-05-17 21:29:54,929 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:54,929 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-05-17 21:29:54,929 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 165 bytes
2016-05-17 21:29:54,929 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CE5A74EB17C0417CBAAB125A95AF7BF8
2016-05-17 21:29:54,929 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-05-17 21:29:54,929 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B73C7AD560684BFDB1B5AE04B2DD7C77, all 1 inputs ready in: 00:00:00.001
2016-05-17 21:29:54,931 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:54,933 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.KeyTupleDeserializer@4d543f6f; valueDeserializer=cascading.tuple.hadoop.io.ValueTupleDeserializer@5610b812
2016-05-17 21:29:54,933 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:54,934 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(91)) - caught throwable
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:54,934 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-05-17 21:29:54,935 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-05-17 21:29:54,935 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1463520594671_0001_1_01_000000_0
2016-05-17 21:29:54,935 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1463520594671_0001_1_01_000000_0 given a go for committing the task output.
2016-05-17 21:29:54,936 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1463520594671_0001_1_01_000000_0
cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
2016-05-17 21:29:54,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1463520594930, Process_Duration=6, Process_End_Time=1463520594936, Read_Duration=0, Tuples_Read=1]]
2016-05-17 21:29:54,938 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B73C7AD560684BFDB1B5AE04B2DD7C77, taskAttemptId=attempt_1463520594671_0001_1_01_000000_0, startTime=1463520594904, finishTime=1463520594938, timeTaken=34, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=213, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=181, SHUFFLE_BYTES_DECOMPRESSED=177, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=181, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520594930, Process_Duration=6, Process_End_Time=1463520594936, Read_Duration=0, Tuples_Read=1
2016-05-17 21:29:54,939 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520594671_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-05-17 21:29:54,939 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1463520594671_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-05-17 21:29:54,939 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:TASK_FINISHED]: vertexName=B73C7AD560684BFDB1B5AE04B2DD7C77, taskId=task_1463520594671_0001_1_01_000000, startTime=1463520594904, finishTime=1463520594939, timeTaken=35, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
], counters=Counters: 0
2016-05-17 21:29:54,939 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1463520594671_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-05-17 21:29:54,939 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77] : 1
2016-05-17 21:29:54,939 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77] because task failed: task_1463520594671_0001_1_01_000000
2016-05-17 21:29:54,939 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77] due to trigger: OWN_TASK_FAILURE
2016-05-17 21:29:54,939 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-05-17 21:29:54,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-05-17 21:29:54,938 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B73C7AD560684BFDB1B5AE04B2DD7C77, mem on close (mb), free: 171, total: 366, max: 672
2016-05-17 21:29:54,939 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-05-17 21:29:54,939 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77]
2016-05-17 21:29:54,940 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-05-17 21:29:54,940 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=F291B28A9E234D1491D4E8F1231CE331, vertexId=vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77]
2016-05-17 21:29:54,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=B73C7AD560684BFDB1B5AE04B2DD7C77, index=1
2016-05-17 21:29:54,941 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CE5A74EB17C0417CBAAB125A95AF7BF8
2016-05-17 21:29:54,941 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:VERTEX_FINISHED]: vertexName=B73C7AD560684BFDB1B5AE04B2DD7C77, vertexId=vertex_1463520594671_0001_1_01, initRequestedTime=1463520594828, initedTime=1463520594835, startRequestedTime=1463520594839, startedTime=1463520594839, finishTime=1463520594941, timeTaken=102, status=FAILED, diagnostics=Task failed, taskId=task_1463520594671_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1463520594904, firstTasksToStart=[ task_1463520594671_0001_1_01_000000 ], lastTaskFinishTime=1463520594938, lastTasksToFinish=[ task_1463520594671_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-05-17 21:29:54,942 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-05-17 21:29:54,942 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1463520594671_0001_00_000002, stoppedTime=1463520594942, exitStatus=0
2016-05-17 21:29:54,942 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594671_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-05-17 21:29:54,942 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1463520594671_0001_1_01_000000_0 failed: true container: container_1463520594671_0001_00_000002 numFailedTAs: 0
2016-05-17 21:29:54,942 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-05-17 21:29:54,942 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-05-17 21:29:54,942 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1463520594671_0001_1
2016-05-17 21:29:54,942 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: F291B28A9E234D1491D4E8F1231CE331 for vertex: vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77]
2016-05-17 21:29:54,943 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1463520594671_0001_1_00 [CE5A74EB17C0417CBAAB125A95AF7BF8]
2016-05-17 21:29:54,941 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-05-17 21:29:54,941 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-05-17 21:29:54,943 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-05-17 21:29:54,944 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1463520594671_0001_1, queueSize=0
2016-05-17 21:29:54,944 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-05-17 21:29:54,944 INFO util.BaseTupleComparator (BaseTupleComparator.java:setConf(57)) - enabling raw byte comparison and ordering
2016-05-17 21:29:54,945 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-05-17 21:29:54,945 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-05-17 21:29:54,945 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1463520594671_0001_1][Event:DAG_FINISHED]: dagId=dag_1463520594671_0001_1, startTime=1463520594827, finishTime=1463520594943, timeTaken=116, status=FAILED, diagnostics=Vertex failed, vertexName=B73C7AD560684BFDB1B5AE04B2DD7C77, vertexId=vertex_1463520594671_0001_1_01, diagnostics=[Task failed, taskId=task_1463520594671_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.pipe.OperatorException: [every][cascading.detail.EveryAssemblyFactory.createAssembly(EveryAssemblyFactory.java:36)] operator Every failed completing operation: TestAggregator[decl:ARGS]
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:161)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:147)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:83)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:114)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: current entry and given tuple must be same length
at cascading.tuple.TupleEntry.setCanonicalTuple(TupleEntry.java:405)
at cascading.TestAggregator.complete(TestAggregator.java:112)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
... 19 more
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1463520594671_0001_1_01 [B73C7AD560684BFDB1B5AE04B2DD7C77] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=213, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=383778816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=131, OUTPUT_BYTES_WITH_OVERHEAD=177, OUTPUT_BYTES_PHYSICAL=181, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1463520594886, Process_Duration=2, Process_End_Time=1463520594888, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-05-17 21:29:54,945 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1463520594671_0001_1 finished with state: FAILED
2016-05-17 21:29:54,945 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1463520594671_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-05-17 21:29:54,945 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1463520594671_0001_00_000002. Relying on regular task shutdown for it to end
2016-05-17 21:29:54 Completed Dag: dag_1463520594671_0001_1
2016-05-17 21:29:54,946 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1463520594671_0001_1, dagState=FAILED
2016-05-17 21:29:54,946 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1463520594671_0001_1_post
2016-05-17 21:29:54,946 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-05-17 21:29:54,946 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-05-17 21:29:54,946 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594671_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-05-17 21:29:54,945 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-05-17 21:29:54,946 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-05-17 21:29:54,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=B73C7AD560684BFDB1B5AE04B2DD7C77, sourceVertex=CE5A74EB17C0417CBAAB125A95AF7BF8
2016-05-17 21:29:54,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=B73C7AD560684BFDB1B5AE04B2DD7C77, sourceVertex=F291B28A9E234D1491D4E8F1231CE331
2016-05-17 21:29:54,947 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1463520594671_0001_00_000002
2016-05-17 21:29:54,947 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1463520594671_0001_00_000002
2016-05-17 21:29:54,947 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1463520594671_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-05-17 21:29:54,947 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1463520594671_0001_00_000002 completed but with errors
2016-05-17 21:29:54,947 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1463520594671_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-05-17 21:29:54,947 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1463520594671_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-05-17 21:29:54,947 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1463520594671_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-05-17 21:29:54,952 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/ALL_ARGS_SWAP, applicationId=application_1463520594671_0001
2016-05-17 21:29:54,953 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-05-17 21:29:54,953 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-05-17 21:29:54,953 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-05-17 21:29:54,953 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-05-17 21:29:54,953 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-05-17 21:29:54,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-05-17 21:29:54,954 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-05-17 21:29:54,954 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-05-17 21:29:54,954 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-05-17 21:29:54,954 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-05-17 21:29:54,954 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-05-17 21:29:54,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-05-17 21:29:54,954 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-05-17 21:29:54,957 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/tmp/.staging/E3753DA17CE04D928BBD580BE2C1220C/E3CFB70A618241E1823F8AB2D7C8BE36/.tez/application_1463520594671_0001
2016-05-17 21:29:54,957 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping all jobs
2016-05-17 21:29:54,958 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopping: (1/1) ...blyplatform/ALL_ARGS_SWAP
2016-05-17 21:29:54,958 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] stopped all jobs
2016-05-17 21:29:54,958 INFO util.Hadoop18TapUtil (Hadoop18TapUtil.java:cleanTempPath(226)) - deleting temp path /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_SWAP/_temporary
2016-05-17 21:29:54,959 INFO flow.Flow (BaseFlow.java:logInfo(1504)) - [every] completed in: 00:00.303
2016-05-17 21:29:54,959 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/2f4c45215847827d/apache-2.4.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_SWAP